Sep 19 12:23:06.735846 ip-10-0-131-60 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 19 12:23:06.736039 ip-10-0-131-60 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 19 12:23:06.736127 ip-10-0-131-60 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 19 12:23:06.736607 ip-10-0-131-60 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 19 12:23:16.983296 ip-10-0-131-60 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 19 12:23:16.983312 ip-10-0-131-60 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 7518f25e13c74ac69c328445711ccd22 --
Sep 19 12:25:15.254755 ip-10-0-131-60 systemd[1]: Starting Kubernetes Kubelet...
Sep 19 12:25:18.551746 ip-10-0-131-60 kubenswrapper[2571]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 12:25:18.551746 ip-10-0-131-60 kubenswrapper[2571]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 19 12:25:18.551746 ip-10-0-131-60 kubenswrapper[2571]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 12:25:18.551746 ip-10-0-131-60 kubenswrapper[2571]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 19 12:25:18.551746 ip-10-0-131-60 kubenswrapper[2571]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 19 12:25:18.552613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.551819    2571 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 19 12:25:18.556095 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556080    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:18.556095 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556094    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556098    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556101    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556104    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556107    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556111    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556114    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556116    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556119    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556121    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556124    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556127    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556133    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556136    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556139    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556141    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556144    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556146    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556150    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556152    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:18.556158 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556155    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556157    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556160    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556164    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556167    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556170    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556172    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556175    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556178    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556180    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556185    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556188    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556193    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556196    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556199    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556201    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556204    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556206    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556211    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:18.556649 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556214    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556217    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556220    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556223    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556225    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556228    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556231    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556234    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556236    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556239    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556241    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556244    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556247    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556249    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556253    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556258    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556261    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556264    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556266    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556269    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:18.557133 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556272    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556274    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556277    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556280    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556282    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556285    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556288    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556291    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556294    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556296    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556299    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556302    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556305    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556307    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556310    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556313    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556315    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556318    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556321    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:18.557639 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556323    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556326    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556328    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556331    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556333    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556336    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556338    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556341    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556345    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556348    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556351    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556353    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556356    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556358    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556361    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556363    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556366    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556369    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556371    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.556374    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:18.558124 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557041    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557047    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557050    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557053    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557056    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557060    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557063    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557066    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557068    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557071    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557073    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557076    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557079    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557082    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557084    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557087    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557089    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557092    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557095    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557099    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:18.558632 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557102    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557104    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557107    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557110    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557112    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557115    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557118    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557120    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557125    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557128    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557131    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557134    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557137    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557140    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557143    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557146    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557148    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557151    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557154    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557157    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:18.559116 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557161    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557164    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557167    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557169    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557172    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557174    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557177    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557179    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557182    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557185    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557187    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557190    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557193    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557195    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557198    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557200    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557203    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557206    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557209    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:18.559619 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557211    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557214    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557216    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557219    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557222    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557224    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557227    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557229    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557232    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557234    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557237    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557239    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557248    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557251    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557255    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557258    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557260    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557263    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557266    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:18.560081 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557269    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557272    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557274    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557277    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557280    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557283    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557286    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557289    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557292    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557294    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557297    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557300    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557302    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557305    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557307    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557310    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557312    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557315    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557318    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557321    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:18.560555 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557324    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557686    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557691    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557693    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557696    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557699    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557704    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557708    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557711    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557714    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557716    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557719    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557721    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557724    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557727    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557729    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557732    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557734    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557737    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:18.561034 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557740    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557743    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557746    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557749    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557751    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557754    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557756    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557759    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557763    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557766    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557769    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557772    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557775    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557777    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557780    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557783    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557785    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557788    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557791    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:18.561548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557794    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557797    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557799    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557802    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557804    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557807    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557809    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557813    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557816    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557818    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557821    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557823    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557826    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557828    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557831    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557834    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557836    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557839    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557842    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557845    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557847    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:18.562008 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557850    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557852    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557855    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557857    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557860    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557863    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557865    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557868    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557871    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557874    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557876    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557879    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557882    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557885    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557888    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557890    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557893    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557895    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557898    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:18.562550 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557900    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557903    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557906    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557908    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557911    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557914    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557916    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557919    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557921    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557924    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557927    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557929    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557932    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557935    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557937    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557939    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557942    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557944    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557947    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557949    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:18.562998 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557952    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.557954    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558025    2571 flags.go:64] FLAG: --address="0.0.0.0"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558064    2571 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558070    2571 flags.go:64] FLAG: --anonymous-auth="true"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558075    2571 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558079    2571 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558083    2571 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558087    2571 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558090    2571 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558094    2571 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558097    2571 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558100    2571 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558103    2571 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558107    2571 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558112    2571 flags.go:64] FLAG: --cgroup-root=""
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558115    2571 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558118    2571 flags.go:64] FLAG: --client-ca-file=""
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558121    2571 flags.go:64] FLAG: --cloud-provider="external"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558124    2571 flags.go:64] FLAG: --cluster-dns="[]"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558129    2571 flags.go:64] FLAG: --cluster-domain=""
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558134    2571 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558138    2571 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558141    2571 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 19 12:25:18.563479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558144    2571 flags.go:64] FLAG: --container-log-max-files="5"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558148    2571 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558151    2571 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558154    2571 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558158    2571 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558161    2571 flags.go:64] FLAG: --contention-profiling="false"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558164    2571 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558167    2571 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558170    2571 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558173    2571 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558177    2571 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558180    2571 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558183    2571 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558186    2571 flags.go:64] FLAG: --enable-load-reader="false"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558189    2571 flags.go:64] FLAG: --enable-server="true"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558193    2571 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558197    2571 flags.go:64] FLAG: --event-burst="100"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558200    2571 flags.go:64] FLAG: --event-qps="50"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558203    2571 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558206    2571 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558209    2571 flags.go:64] FLAG: --eviction-hard=""
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558213    2571 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558216    2571 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558220    2571 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558223    2571 flags.go:64] FLAG: --eviction-soft=""
Sep 19 12:25:18.564052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558226    2571 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558229    2571 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558232    2571 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558235    2571 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558239    2571 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558243    2571 flags.go:64] FLAG: --fail-swap-on="true"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558246    2571 flags.go:64] FLAG: --feature-gates=""
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558250    2571 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558253    2571 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558256    2571 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558260    2571 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558263    2571 flags.go:64] FLAG: --healthz-port="10248"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558275    2571 flags.go:64] FLAG: --help="false"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558278    2571 flags.go:64] FLAG: --hostname-override="ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558282    2571 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558285    2571 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558288    2571 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558291    2571 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558295    2571 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558298    2571 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558300    2571 flags.go:64] FLAG: --image-service-endpoint=""
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558304    2571 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558306    2571 flags.go:64] FLAG: --kube-api-burst="100"
Sep 19 12:25:18.564668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558310    2571 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558313    2571 flags.go:64] FLAG: --kube-api-qps="50"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558316    2571 flags.go:64] FLAG: --kube-reserved=""
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558319    2571 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558323    2571 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558326    2571 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558328    2571 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558333    2571 flags.go:64] FLAG: --lock-file=""
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558336    2571 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558339    2571 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558342    2571 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558347    2571 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558350    2571 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558354    2571 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558357    2571 flags.go:64] FLAG: --logging-format="text"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558360    2571 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558363    2571 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558366    2571 flags.go:64] FLAG: --manifest-url=""
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558369    2571 flags.go:64] FLAG: --manifest-url-header=""
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558376    2571 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558379    2571 flags.go:64] FLAG: --max-open-files="1000000"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558383    2571 flags.go:64] FLAG: --max-pods="110"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558386    2571 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558389    2571 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558392    2571 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 19 12:25:18.565291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558395    2571 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558398    2571 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558401    2571 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558404    2571 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558411    2571 flags.go:64] FLAG: --node-status-max-images="50"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558414    2571 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558417    2571 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558420    2571 flags.go:64] FLAG: --pod-cidr=""
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558424    2571 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558429    2571 flags.go:64] FLAG: --pod-manifest-path=""
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558432    2571 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558435    2571 flags.go:64] FLAG: --pods-per-core="0"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558438    2571 flags.go:64] FLAG: --port="10250"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558441    2571 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558444    2571 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c749d2b53b0b517a"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558448    2571 flags.go:64] FLAG: --qos-reserved=""
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558451    2571 flags.go:64] FLAG: --read-only-port="10255"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558454    2571 flags.go:64] FLAG: --register-node="true"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558469    2571 flags.go:64] FLAG: --register-with-taints=""
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558473    2571 flags.go:64] FLAG: --registry-burst="10"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558477    2571 flags.go:64] FLAG: --registry-qps="5"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558480    2571 flags.go:64] FLAG: --reserved-cpus=""
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558483    2571 flags.go:64] FLAG: --reserved-memory=""
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558487    2571 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558490    2571 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 19 12:25:18.565968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558493    2571 flags.go:64] FLAG: --rotate-certificates="false"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558496    2571 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558499    2571 flags.go:64] FLAG: --runonce="false"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558502    2571 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558505    2571 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558508    2571 flags.go:64] FLAG: --seccomp-default="false"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558511    2571 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558513    2571 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558517    2571 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558520    2571 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558523    2571 flags.go:64] FLAG: --storage-driver-password="root"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558525    2571 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558528    2571 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558531    2571 flags.go:64] FLAG: --storage-driver-user="root"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558534    2571 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558537    2571 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558541    2571 flags.go:64] FLAG: --system-cgroups=""
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558543    2571 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558549    2571 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558552    2571 flags.go:64] FLAG: --tls-cert-file=""
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558554    2571 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558558    2571 flags.go:64] FLAG: --tls-min-version=""
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558562    2571 flags.go:64] FLAG: --tls-private-key-file=""
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558565    2571 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558568    2571 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 19 12:25:18.566590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558571    2571 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558574    2571 flags.go:64] FLAG: --v="2"
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558579    2571 flags.go:64] FLAG: --version="false"
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558583    2571 flags.go:64] FLAG: --vmodule=""
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558587    2571 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.558590    2571 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558706    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558710    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558713    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558716    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558718    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558721    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558724    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558726    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558729    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558732    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558735    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558737    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558740    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558743    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558745    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:18.567206 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558749    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558753    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558756    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558760    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558763    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558766    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558768    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558771    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558775    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558778    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558780    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558783    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558786    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558790    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558793    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558796    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558799    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558801    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558804    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558806    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:18.567746 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558809    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558812    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558814    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558817    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558820    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558822    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558825    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558827    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558830    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558832    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558835    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558838    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558840    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558843    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558846    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558848    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558855    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558859    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558862    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558865    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:18.568270 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558869    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558872    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558875    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558878    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558881    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558885    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558887    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558890    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558893    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558895    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558898    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558901    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558903    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558906    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558909    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558913    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558916    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558919    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558922    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:18.568769 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558925    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558928    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558931    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558934    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558936    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558939    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558941    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558944    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558947    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558950    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558953    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558955    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558958    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558962    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558965    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558967    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558970    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558973    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558977    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558980    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:18.569226 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558982    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558985    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558988    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558990    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.558993    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.559409    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.566526    2571 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.566540    2571 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.566559    2571 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.566565    2571 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566617    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566622    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566626    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566629    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:18.569771 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566632    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566635    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566639    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566641    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566644    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566647    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566649    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566652    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566656    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566659    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566662    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566664    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566667    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566670    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566672    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566675    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566677    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566680    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566683    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566685    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:18.570121 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566688    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566690    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566693    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566696    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566699    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566702    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566705    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566708    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566710    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566713    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566715    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566718    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566721    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566723    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566726    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566729    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566731    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566733    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566736    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566739    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:18.570762 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566741    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566744    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566748    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566752    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566755    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566758    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566760    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566763    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566765    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566768    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566771    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566773    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566776    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566779    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566781    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566784    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566787    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566790    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566793    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566796    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:18.571597 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566798    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566801    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566803    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566806    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566809    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566812    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566814    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566818    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566822    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566826    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566830    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566833    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566836    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566839    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566842    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566845    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566848    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566851    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566853    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:18.572117 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566856    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566859    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566862    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566864    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566866    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566869    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566871    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566874    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566877    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566879    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566882    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566885    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566888    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566890    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566893    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:18.572695 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.566896    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.566900    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567007    2571 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567012    2571 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567016    2571 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567019    2571 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567022    2571 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567025    2571 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567028    2571 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567031    2571 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567033    2571 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567036    2571 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567039    2571 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567042    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 19 12:25:18.573051 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567045    2571 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567048    2571 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567051    2571 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567053    2571 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567056    2571 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567059    2571 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567061    2571 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567064    2571 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567066    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567069    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567072    2571 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567074    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567077    2571 feature_gate.go:328] unrecognized feature gate: Example2
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567079    2571 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567082    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567085    2571 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567087    2571 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567090    2571 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567092    2571 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567095    2571 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 19 12:25:18.573445 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567097    2571 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567100    2571 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567102    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567105    2571 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567107    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567110    2571 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567112    2571 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567115    2571 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567117    2571 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567121    2571 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567124    2571 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567127    2571 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567130    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567133    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567135    2571 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567138    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567140    2571 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567143    2571 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567146    2571 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567148    2571 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 19 12:25:18.574248 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567150    2571 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567153    2571 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567157    2571 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567160    2571 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567163    2571 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567166    2571 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567169    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567172    2571 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567175    2571 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567177    2571 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567180    2571 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567183    2571 feature_gate.go:328] unrecognized feature gate: Example
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567185    2571 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567188    2571 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567191    2571 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567193    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567196    2571 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567198    2571 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567201    2571 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 19 12:25:18.574952 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567204    2571 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567207    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567210    2571 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567212    2571 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567215    2571 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567217    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567221    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567223    2571 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567226    2571 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567229    2571 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567231    2571 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567234    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567236    2571 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567239    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567242    2571 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567244    2571 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567247    2571 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567249    2571 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567252    2571 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567255    2571 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 19 12:25:18.575412 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567257    2571 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567260    2571 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567262    2571 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567265    2571 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567268    2571 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567270    2571 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567273    2571 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:18.567275    2571 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.567280    2571 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.567372    2571 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.569376    2571 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.570352    2571 server.go:1019] "Starting client certificate rotation"
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.570484    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 12:25:18.575916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.570535    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 19 12:25:18.576255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.574204    2571 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 12:25:18.576314 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.576300    2571 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 19 12:25:18.588053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.588037    2571 log.go:25] "Validated CRI v1 runtime API"
Sep 19 12:25:18.589661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.589645    2571 log.go:25] "Validated CRI v1 image API"
Sep 19 12:25:18.591504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.591483    2571 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 19 12:25:18.593700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.593683    2571 fs.go:135] Filesystem UUIDs: map[21735e87-084f-43e8-a6e1-d670a0894ac2:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 c274d5a7-80dc-48e1-a5cd-3579ce86af0f:/dev/nvme0n1p4]
Sep 19 12:25:18.593758 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.593700    2571 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 19 12:25:18.598107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.597996    2571 manager.go:216] Machine: {Timestamp:2026-09-19 12:25:18.59750067 +0000 UTC m=+1.957743603 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100412 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27c14549b0d6b15f650083e4c0d0c2 SystemUUID:ec27c145-49b0-d6b1-5f65-0083e4c0d0c2 BootID:7518f25e-13c7-4ac6-9c32-8445711ccd22 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e5:f9:c2:53 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e5:f9:c2:53 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:38:01:79:19:54 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 19 12:25:18.598107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.598101    2571 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 19 12:25:18.598210 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.598201    2571 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 19 12:25:18.599671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.599644    2571 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 19 12:25:18.599800 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.599675    2571 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-60.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 19 12:25:18.600802 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.600791    2571 topology_manager.go:138] "Creating topology manager with none policy"
Sep 19 12:25:18.600839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.600806    2571 container_manager_linux.go:309] "Creating device plugin manager"
Sep 19 12:25:18.600839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.600814    2571 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 12:25:18.600839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.600824    2571 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 19 12:25:18.602006 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.601996    2571 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 19 12:25:18.604612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604601    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 12:25:18.604733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604723    2571 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 19 12:25:18.604761 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604754    2571 kubelet.go:486] "Attempting to sync node with API server"
Sep 19 12:25:18.604794 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604767    2571 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 19 12:25:18.604794 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604784    2571 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 19 12:25:18.604794 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604793    2571 kubelet.go:398] "Adding apiserver pod source"
Sep 19 12:25:18.604875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.604802    2571 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.605185    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.605210    2571 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.605723    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.607072    2571 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609026    2571 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609053    2571 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609918    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609931    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609937    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609942    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609951    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609957    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609971    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.609976    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.610167    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.610257    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.610286    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.610294    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.610317    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 19 12:25:18.610482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.610323    2571 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 19 12:25:18.614141 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.614116    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 19 12:25:18.614141 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.614116    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-60.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 19 12:25:18.614252 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.614160    2571 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-60.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 19 12:25:18.614989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.614975    2571 server.go:1268] "Started kubelet"
Sep 19 12:25:18.615087 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.615059    2571 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 19 12:25:18.615148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.615056    2571 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 19 12:25:18.615148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.615122    2571 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 19 12:25:18.615945 ip-10-0-131-60 systemd[1]: Started Kubernetes Kubelet.
Sep 19 12:25:18.616604 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.616266    2571 server.go:310] "Adding debug handlers to kubelet server"
Sep 19 12:25:18.616958 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.616922    2571 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 19 12:25:18.620627 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.620600    2571 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 19 12:25:18.623322 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.623163    2571 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 19 12:25:18.623407 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.623175    2571 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 19 12:25:18.623574 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.623553    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:18.624131 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624113    2571 factory.go:55] Registering systemd factory
Sep 19 12:25:18.624131 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624130    2571 factory.go:223] Registration of the systemd container factory successfully
Sep 19 12:25:18.624271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624186    2571 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 19 12:25:18.624271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624184    2571 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 19 12:25:18.624271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624252    2571 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 19 12:25:18.624387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624332    2571 factory.go:153] Registering CRI-O factory
Sep 19 12:25:18.624387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624343    2571 factory.go:223] Registration of the crio container factory successfully
Sep 19 12:25:18.624387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624347    2571 reconstruct.go:97] "Volume reconstruction finished"
Sep 19 12:25:18.624387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624356    2571 reconciler.go:29] "Reconciler: start to sync state"
Sep 19 12:25:18.624547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624442    2571 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 19 12:25:18.624547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624517    2571 factory.go:103] Registering Raw factory
Sep 19 12:25:18.624547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624534    2571 manager.go:1196] Started watching for new ooms in manager
Sep 19 12:25:18.624935 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.624924    2571 manager.go:349] Starting recovery of all containers
Sep 19 12:25:18.625732 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.625711    2571 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-60.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 19 12:25:18.625809 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.625788    2571 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 19 12:25:18.626350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.626333    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b26dv"
Sep 19 12:25:18.633591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.633568    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-b26dv"
Sep 19 12:25:18.635168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.635155    2571 manager.go:354] Recovery completed
Sep 19 12:25:18.638940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.638928    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.640557 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.640543    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.640620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.640568    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.640620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.640577    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.641075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641062    2571 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 19 12:25:18.641075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641073    2571 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 19 12:25:18.641157 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641083    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.641157 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641086    2571 state_mem.go:36] "Initialized new in-memory state store"
Sep 19 12:25:18.641768 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641754    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.641817 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641778    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.641817 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.641787    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.642216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.642205    2571 policy_none.go:49] "None policy: Start"
Sep 19 12:25:18.642247 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.642221    2571 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 19 12:25:18.642247 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.642230    2571 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 19 12:25:18.646831 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.646811    2571 policy_none.go:47] "Start"
Sep 19 12:25:18.692038 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692018    2571 manager.go:337] "Starting Device Plugin manager"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.692113    2571 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692123    2571 server.go:87] "Starting device plugin registration server"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692378    2571 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692396    2571 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692510    2571 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692590    2571 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.692599    2571 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.693095    2571 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 19 12:25:18.697621 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.693136    2571 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:18.731960 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.731927    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 19 12:25:18.733126 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.733109    2571 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 19 12:25:18.733173 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.733165    2571 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 19 12:25:18.733212 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.733195    2571 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 19 12:25:18.733264 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.733241    2571 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 19 12:25:18.734952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.734930    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:18.792824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.792784    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.793827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.793805    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.793952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.793843    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.793952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.793858    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.793952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.793890    2571 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.803014 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.802964    2571 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.803014 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.802989    2571 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-60.ec2.internal\": node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:18.821674 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.821656    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:18.834343 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.834309    2571 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"]
Sep 19 12:25:18.834415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.834380    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.836076 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.836062    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.836135 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.836090    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.836135 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.836100    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.838281 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.838269    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.838386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.838372    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.838424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.838406    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.839032 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.839016    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.839103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.839038    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.839103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.839048    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.839103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.839061    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.839103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.839064    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.839103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.839077    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.841213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.841198    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.841272 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.841224    2571 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 19 12:25:18.841921 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.841907    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientMemory"
Sep 19 12:25:18.841989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.841932    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasNoDiskPressure"
Sep 19 12:25:18.841989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.841946    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeHasSufficientPID"
Sep 19 12:25:18.866154 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.866134    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-60.ec2.internal\" not found" node="ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.870597 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.870582    2571 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-60.ec2.internal\" not found" node="ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.922501 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:18.922479    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:18.926866 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.926849    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/769ded29dc5f76d39a7c0031f5b9ed81-config\") pod \"kube-apiserver-proxy-ip-10-0-131-60.ec2.internal\" (UID: \"769ded29dc5f76d39a7c0031f5b9ed81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.926916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.926875    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/794c2de5d33bcf417220a257d18a1d6b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal\" (UID: \"794c2de5d33bcf417220a257d18a1d6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:18.926916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:18.926893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/794c2de5d33bcf417220a257d18a1d6b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal\" (UID: \"794c2de5d33bcf417220a257d18a1d6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.023557 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.023524    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.027782 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.027763    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/794c2de5d33bcf417220a257d18a1d6b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal\" (UID: \"794c2de5d33bcf417220a257d18a1d6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.027861 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.027792    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/794c2de5d33bcf417220a257d18a1d6b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal\" (UID: \"794c2de5d33bcf417220a257d18a1d6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.027861 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.027809    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/769ded29dc5f76d39a7c0031f5b9ed81-config\") pod \"kube-apiserver-proxy-ip-10-0-131-60.ec2.internal\" (UID: \"769ded29dc5f76d39a7c0031f5b9ed81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.027935 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.027871    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/794c2de5d33bcf417220a257d18a1d6b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal\" (UID: \"794c2de5d33bcf417220a257d18a1d6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.027935 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.027922    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/769ded29dc5f76d39a7c0031f5b9ed81-config\") pod \"kube-apiserver-proxy-ip-10-0-131-60.ec2.internal\" (UID: \"769ded29dc5f76d39a7c0031f5b9ed81\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.028008 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.027959    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/794c2de5d33bcf417220a257d18a1d6b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal\" (UID: \"794c2de5d33bcf417220a257d18a1d6b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.124269 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.124195    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.169602 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.169576    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.175301 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.174837    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.224472 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.224427    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.324969 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.324941    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.425429 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.425344    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.525834 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.525793    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.570097 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.570075    2571 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 19 12:25:19.570662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.570207    2571 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 19 12:25:19.626656 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.626626    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.638301 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.638252    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-18 12:20:18 +0000 UTC" deadline="2028-07-02 01:29:20.11577354 +0000 UTC"
Sep 19 12:25:19.638431 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.638298    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15637h4m0.477480384s"
Sep 19 12:25:19.726750 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.726706    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.747474 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:19.747431    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769ded29dc5f76d39a7c0031f5b9ed81.slice/crio-64dbcc25ee8f37b7277b6d127311ae36ab647eb1bc3fec5996cfc07ea16be8b8 WatchSource:0}: Error finding container 64dbcc25ee8f37b7277b6d127311ae36ab647eb1bc3fec5996cfc07ea16be8b8: Status 404 returned error can't find the container with id 64dbcc25ee8f37b7277b6d127311ae36ab647eb1bc3fec5996cfc07ea16be8b8
Sep 19 12:25:19.748021 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:19.748002    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod794c2de5d33bcf417220a257d18a1d6b.slice/crio-70bd6b477f809bb7de1427df20e080022aaff01dd7bb5ff471eb391bec7bf07f WatchSource:0}: Error finding container 70bd6b477f809bb7de1427df20e080022aaff01dd7bb5ff471eb391bec7bf07f: Status 404 returned error can't find the container with id 70bd6b477f809bb7de1427df20e080022aaff01dd7bb5ff471eb391bec7bf07f
Sep 19 12:25:19.752819 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.752805    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:25:19.766954 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.766933    2571 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:19.827314 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:19.827271    2571 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-60.ec2.internal\" not found"
Sep 19 12:25:19.867910 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.867882    2571 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:19.870122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.870105    2571 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:19.924056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.924031    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.932540 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.932520    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 12:25:19.933330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.933315    2571 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal"
Sep 19 12:25:19.942684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:19.942665    2571 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 19 12:25:20.440390 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.440363    2571 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 19 12:25:20.606195 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.606167    2571 apiserver.go:52] "Watching apiserver"
Sep 19 12:25:20.614279 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.614248    2571 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 19 12:25:20.614653 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.614628    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-wf7d9","openshift-dns/node-resolver-qx4xs","openshift-image-registry/node-ca-jpl8l","openshift-multus/multus-additional-cni-plugins-7qjb8","openshift-multus/multus-cxgd2","openshift-network-diagnostics/network-check-target-c2nxz","kube-system/konnectivity-agent-jr2dg","kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x","openshift-cluster-node-tuning-operator/tuned-8gsh2","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal","openshift-multus/network-metrics-daemon-4569q","openshift-network-operator/iptables-alerter-5pfl5","openshift-ovn-kubernetes/ovnkube-node-zpcx5"]
Sep 19 12:25:20.617031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.617005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.619210 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.619179    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.619356 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.619338    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 19 12:25:20.619732 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.619712    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jmmh4\""
Sep 19 12:25:20.619811 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.619712    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 19 12:25:20.621435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.621414    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.621542 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.621510    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-2g2rt\""
Sep 19 12:25:20.621669 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.621608    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.621883 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.621839    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.622075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.622059    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 19 12:25:20.623534 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.623480    2571 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 19 12:25:20.623802 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.623771    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 19 12:25:20.623883 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.623781    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.624250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.624222    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.624342 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.624298    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.624550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.624384    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d9vjp\""
Sep 19 12:25:20.625950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.625925    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.626196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.626178    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 19 12:25:20.626282 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.626260    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.626572 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.626558    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m8g88\""
Sep 19 12:25:20.626645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.626559    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.626811 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.626793    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 19 12:25:20.626811 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.626807    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 19 12:25:20.628086 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.628055    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:20.628176 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.628137    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:20.628328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.628310    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 19 12:25:20.628380 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.628366    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8wf5x\""
Sep 19 12:25:20.630392 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.630315    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.632484 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.632450    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-9k2xd\""
Sep 19 12:25:20.632573 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.632535    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.632640 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.632592    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:20.632797 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.632775    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.632868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.632824    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 19 12:25:20.632868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.632838    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.635227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.635207    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.636112 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.636210 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636128    2571 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 19 12:25:20.636210 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636128    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqsqt\" (UniqueName: \"kubernetes.io/projected/638fdc50-3117-472b-a7f7-133a1c8d015a-kube-api-access-nqsqt\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.636210 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636199    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-k8s-cni-cncf-io\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636225    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-cni-multus\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636252    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-hostroot\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636275    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-cni-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-cni-bin\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636363    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffqh5\" (UniqueName: \"kubernetes.io/projected/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-kube-api-access-ffqh5\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636391    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bjp7\" (UniqueName: \"kubernetes.io/projected/b0eb4745-b388-4c27-aea0-da13c0976513-kube-api-access-6bjp7\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.636672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636418    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.636672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636530    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-daemon-config\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636567    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-etc-kubernetes\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636602    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2291616-999a-4193-8882-f43d2174c60e-host\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.636672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-cni-binary-copy\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-netns\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-kubelet\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636752    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc8vn\" (UniqueName: \"kubernetes.io/projected/e2291616-999a-4193-8882-f43d2174c60e-kube-api-access-nc8vn\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636776    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-cni-binary-copy\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636801    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b0eb4745-b388-4c27-aea0-da13c0976513-host-slash\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636825    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e2291616-999a-4193-8882-f43d2174c60e-serviceca\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636877    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-cnibin\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.636947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.637296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.636969    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-cnibin\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.637296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637005    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-system-cni-dir\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.637296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e407c77-d390-4e67-ae03-f083fa4acf52-agent-certs\") pod \"konnectivity-agent-jr2dg\" (UID: \"0e407c77-d390-4e67-ae03-f083fa4acf52\") " pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.637296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637065    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e407c77-d390-4e67-ae03-f083fa4acf52-konnectivity-ca\") pod \"konnectivity-agent-jr2dg\" (UID: \"0e407c77-d390-4e67-ae03-f083fa4acf52\") " pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.637296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637264    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.637699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637677    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b0eb4745-b388-4c27-aea0-da13c0976513-iptables-alerter-script\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.637753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637739    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-system-cni-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.637823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637766    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-socket-dir-parent\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.637823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637800    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-conf-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.637912 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.637823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-multus-certs\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.639164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.639149    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.639319 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.639287    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m8pst\""
Sep 19 12:25:20.639877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.639741    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.639877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.639777    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.639877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.639826    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.640112 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.640086    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.640981 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.640957    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ghd89\""
Sep 19 12:25:20.642033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.642005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:20.642122 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.642068    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:20.642549 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.642529    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rp2g4\""
Sep 19 12:25:20.642763 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.642688    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 19 12:25:20.642763 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.642733    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 19 12:25:20.643125 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.643101    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 19 12:25:20.643248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.643231    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 19 12:25:20.643443 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.643425    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 19 12:25:20.643534 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.643502    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 19 12:25:20.658960 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.658940    2571 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rrv56"
Sep 19 12:25:20.665855 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.665836    2571 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rrv56"
Sep 19 12:25:20.725392 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.725327    2571 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 19 12:25:20.738091 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738059    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bjp7\" (UniqueName: \"kubernetes.io/projected/b0eb4745-b388-4c27-aea0-da13c0976513-kube-api-access-6bjp7\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.738091 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-etc-kubernetes\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738127    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-kubelet\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738144    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738161    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovnkube-config\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738174    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysctl-d\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738199    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-cni-binary-copy\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-netns\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738262    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-etc-kubernetes\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.738312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738309    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-env-overrides\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738314    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-netns\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738345    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-host\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal" event={"ID":"769ded29dc5f76d39a7c0031f5b9ed81","Type":"ContainerStarted","Data":"64dbcc25ee8f37b7277b6d127311ae36ab647eb1bc3fec5996cfc07ea16be8b8"}
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738403    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b0eb4745-b388-4c27-aea0-da13c0976513-host-slash\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b0eb4745-b388-4c27-aea0-da13c0976513-host-slash\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738454    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e2291616-999a-4193-8882-f43d2174c60e-serviceca\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738505    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-cnibin\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738594    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-cnibin\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5pt2\" (UniqueName: \"kubernetes.io/projected/80304af3-45a3-466b-a4ea-d7ac748811f3-kube-api-access-s5pt2\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738665    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-modprobe-d\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.738710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-kubernetes\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738754    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-cni-binary-copy\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738760    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-slash\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738811    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-node-log\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738836    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e407c77-d390-4e67-ae03-f083fa4acf52-konnectivity-ca\") pod \"konnectivity-agent-jr2dg\" (UID: \"0e407c77-d390-4e67-ae03-f083fa4acf52\") " pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738859    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-conf-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738882    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-multus-certs\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738908    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovn-node-metrics-cert\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738923    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e2291616-999a-4193-8882-f43d2174c60e-serviceca\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738932    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738964    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvrws\" (UniqueName: \"kubernetes.io/projected/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-kube-api-access-mvrws\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738980    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-conf-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.738993    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqsqt\" (UniqueName: \"kubernetes.io/projected/638fdc50-3117-472b-a7f7-133a1c8d015a-kube-api-access-nqsqt\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739021    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-cni-multus\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739024    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-multus-certs\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739047    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae5d7b23-07f5-47d3-af44-d1cb64091bca-kubelet-config\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-run-netns\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.739258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739093    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-var-lib-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739114    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-systemd\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739136    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-lib-modules\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739177    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-cni-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739195    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739194    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-cni-multus\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739221    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-cni-bin\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739263    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-cni-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739261    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/0e407c77-d390-4e67-ae03-f083fa4acf52-konnectivity-ca\") pod \"konnectivity-agent-jr2dg\" (UID: \"0e407c77-d390-4e67-ae03-f083fa4acf52\") " pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739264    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-cni-bin\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739300    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffqh5\" (UniqueName: \"kubernetes.io/projected/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-kube-api-access-ffqh5\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739330    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80304af3-45a3-466b-a4ea-d7ac748811f3-hosts-file\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-daemon-config\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739409    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739431    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-ovn\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739455    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvsvz\" (UniqueName: \"kubernetes.io/projected/5fb3f992-7bca-4383-89a9-97a0eaa45a54-kube-api-access-pvsvz\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.740088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-socket-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739525    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2291616-999a-4193-8882-f43d2174c60e-host\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739554    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-systemd-units\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739578    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6wcx\" (UniqueName: \"kubernetes.io/projected/121c696e-d742-4a44-beb3-2caf250a1b96-kube-api-access-c6wcx\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739601    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-device-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739638    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysctl-conf\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739662    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-var-lib-kubelet\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739677    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e2291616-999a-4193-8882-f43d2174c60e-host\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739686    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-kubelet\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739723    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc8vn\" (UniqueName: \"kubernetes.io/projected/e2291616-999a-4193-8882-f43d2174c60e-kube-api-access-nc8vn\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739725    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-var-lib-kubelet\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739791    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-cni-binary-copy\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739824    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739868    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-cni-bin\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739886    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-daemon-config\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739910    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-sys-fs\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739950    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lv9k5\" (UniqueName: \"kubernetes.io/projected/a9784b34-249f-4f94-9ffb-307a2cece9ea-kube-api-access-lv9k5\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.740905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.739988    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-cnibin\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740019    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740042    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-sys\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740078    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-tuned\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740102    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-cnibin\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740105    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-system-cni-dir\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740148    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-systemd\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740158    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-system-cni-dir\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740179    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-log-socket\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740203    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-tmp\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740231    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e407c77-d390-4e67-ae03-f083fa4acf52-agent-certs\") pod \"konnectivity-agent-jr2dg\" (UID: \"0e407c77-d390-4e67-ae03-f083fa4acf52\") " pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b0eb4745-b388-4c27-aea0-da13c0976513-iptables-alerter-script\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740280    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-system-cni-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740319    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-socket-dir-parent\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740335    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-system-cni-dir\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740350    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.741529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740393    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-multus-socket-dir-parent\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740404    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-k8s-cni-cncf-io\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740429    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-hostroot\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740450    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovnkube-script-lib\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740491    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740514    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-registration-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740537    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80304af3-45a3-466b-a4ea-d7ac748811f3-tmp-dir\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740562    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae5d7b23-07f5-47d3-af44-d1cb64091bca-dbus\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740567    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/638fdc50-3117-472b-a7f7-133a1c8d015a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740583    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-etc-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740582    2571 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740614    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-cni-netd\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740625    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-hostroot\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740632    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740656    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-host-run-k8s-cni-cncf-io\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740698    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysconfig\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.742019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740710    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/638fdc50-3117-472b-a7f7-133a1c8d015a-cni-binary-copy\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.742668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-run\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.742668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.740875    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b0eb4745-b388-4c27-aea0-da13c0976513-iptables-alerter-script\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.742668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.741156    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal" event={"ID":"794c2de5d33bcf417220a257d18a1d6b","Type":"ContainerStarted","Data":"70bd6b477f809bb7de1427df20e080022aaff01dd7bb5ff471eb391bec7bf07f"}
Sep 19 12:25:20.744018 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.743998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/0e407c77-d390-4e67-ae03-f083fa4acf52-agent-certs\") pod \"konnectivity-agent-jr2dg\" (UID: \"0e407c77-d390-4e67-ae03-f083fa4acf52\") " pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.749434 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.749264    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqsqt\" (UniqueName: \"kubernetes.io/projected/638fdc50-3117-472b-a7f7-133a1c8d015a-kube-api-access-nqsqt\") pod \"multus-additional-cni-plugins-7qjb8\" (UID: \"638fdc50-3117-472b-a7f7-133a1c8d015a\") " pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.749893 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.749864    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bjp7\" (UniqueName: \"kubernetes.io/projected/b0eb4745-b388-4c27-aea0-da13c0976513-kube-api-access-6bjp7\") pod \"iptables-alerter-5pfl5\" (UID: \"b0eb4745-b388-4c27-aea0-da13c0976513\") " pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.750726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.750704    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffqh5\" (UniqueName: \"kubernetes.io/projected/c00d1258-add9-4b09-a9a6-4d6c82f01ec1-kube-api-access-ffqh5\") pod \"multus-cxgd2\" (UID: \"c00d1258-add9-4b09-a9a6-4d6c82f01ec1\") " pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.751265 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.751248    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc8vn\" (UniqueName: \"kubernetes.io/projected/e2291616-999a-4193-8882-f43d2174c60e-kube-api-access-nc8vn\") pod \"node-ca-jpl8l\" (UID: \"e2291616-999a-4193-8882-f43d2174c60e\") " pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.841328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841293    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.841328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovnkube-script-lib\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841352    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:20.841582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841377    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-registration-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.841582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841432    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.841582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841448    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-registration-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.841582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841528    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80304af3-45a3-466b-a4ea-d7ac748811f3-tmp-dir\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.841582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841574    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae5d7b23-07f5-47d3-af44-d1cb64091bca-dbus\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841599    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-etc-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841624    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-cni-netd\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841672    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-cni-netd\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841685    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841699    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-etc-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841728    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysconfig\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841754    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-run\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841756    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841759    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae5d7b23-07f5-47d3-af44-d1cb64091bca-dbus\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841779    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-kubelet\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841809    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841806    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysconfig\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841816    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-run\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841832    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovnkube-config\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841850    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-kubelet\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841857    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysctl-d\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.841865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841860    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-run-ovn-kubernetes\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841891    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-env-overrides\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841914    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-host\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841944    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5pt2\" (UniqueName: \"kubernetes.io/projected/80304af3-45a3-466b-a4ea-d7ac748811f3-kube-api-access-s5pt2\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841950    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysctl-d\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841979    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-modprobe-d\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.841996    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovnkube-script-lib\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842023    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-host\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842047    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-kubernetes\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842077    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-slash\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842100    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-node-log\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842052    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-modprobe-d\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovn-node-metrics-cert\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842138    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80304af3-45a3-466b-a4ea-d7ac748811f3-tmp-dir\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842155    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842158    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-slash\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842171    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-kubernetes\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842178    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvrws\" (UniqueName: \"kubernetes.io/projected/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-kube-api-access-mvrws\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.842720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842179    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-node-log\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842211    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae5d7b23-07f5-47d3-af44-d1cb64091bca-kubelet-config\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842235    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-run-netns\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.842260    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842297    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-var-lib-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842303    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovnkube-config\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842260    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-var-lib-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842305    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae5d7b23-07f5-47d3-af44-d1cb64091bca-kubelet-config\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.842342    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:21.342321913 +0000 UTC m=+4.702564847 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842362    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-systemd\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842384    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5fb3f992-7bca-4383-89a9-97a0eaa45a54-env-overrides\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842342    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-run-netns\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842398    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-lib-modules\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842429    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-systemd\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842444    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80304af3-45a3-466b-a4ea-d7ac748811f3-hosts-file\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842487    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842501    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/80304af3-45a3-466b-a4ea-d7ac748811f3-hosts-file\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.843550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842508    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-lib-modules\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842512    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-ovn\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842554    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-ovn\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842567    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvsvz\" (UniqueName: \"kubernetes.io/projected/5fb3f992-7bca-4383-89a9-97a0eaa45a54-kube-api-access-pvsvz\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.842587    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842593    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-socket-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.842628    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret podName:ae5d7b23-07f5-47d3-af44-d1cb64091bca nodeName:}" failed. No retries permitted until 2026-09-19 12:25:21.342614047 +0000 UTC m=+4.702856979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret") pod "global-pull-secret-syncer-wf7d9" (UID: "ae5d7b23-07f5-47d3-af44-d1cb64091bca") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842617    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-systemd-units\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842657    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6wcx\" (UniqueName: \"kubernetes.io/projected/121c696e-d742-4a44-beb3-2caf250a1b96-kube-api-access-c6wcx\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842661    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-systemd-units\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842683    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-device-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842707    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysctl-conf\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842743    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-var-lib-kubelet\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842780    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842744    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-socket-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-cni-bin\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842782    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-device-dir\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.844336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842817    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-var-lib-kubelet\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842828    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-sys-fs\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842836    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-openvswitch\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842860    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-host-cni-bin\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842852    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lv9k5\" (UniqueName: \"kubernetes.io/projected/a9784b34-249f-4f94-9ffb-307a2cece9ea-kube-api-access-lv9k5\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842899    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-sys-fs\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842894    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-sysctl-conf\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842924    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842965    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-sys\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.842994    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-tuned\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843019    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-systemd\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843033    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a9784b34-249f-4f94-9ffb-307a2cece9ea-etc-selinux\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843043    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-log-socket\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843068    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-tmp\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843068    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-sys\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843076    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-run-systemd\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.843131    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5fb3f992-7bca-4383-89a9-97a0eaa45a54-log-socket\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.845259    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5fb3f992-7bca-4383-89a9-97a0eaa45a54-ovn-node-metrics-cert\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.845646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.845420    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-etc-tuned\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.845646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.845452    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-tmp\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.847716 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.847698    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:20.847716 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.847717    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:20.847863 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.847726    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jpkqb for pod openshift-network-diagnostics/network-check-target-c2nxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:20.847863 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:20.847791    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb podName:2d320379-f1e7-4f73-ba51-e2559ceb1a1e nodeName:}" failed. No retries permitted until 2026-09-19 12:25:21.347775628 +0000 UTC m=+4.708018552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jpkqb" (UniqueName: "kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb") pod "network-check-target-c2nxz" (UID: "2d320379-f1e7-4f73-ba51-e2559ceb1a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:20.851401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.851365    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvsvz\" (UniqueName: \"kubernetes.io/projected/5fb3f992-7bca-4383-89a9-97a0eaa45a54-kube-api-access-pvsvz\") pod \"ovnkube-node-zpcx5\" (UID: \"5fb3f992-7bca-4383-89a9-97a0eaa45a54\") " pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:20.851401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.851373    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvrws\" (UniqueName: \"kubernetes.io/projected/a7e9fe70-c0cb-4715-82d6-4e4d529484c7-kube-api-access-mvrws\") pod \"tuned-8gsh2\" (UID: \"a7e9fe70-c0cb-4715-82d6-4e4d529484c7\") " pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.851562 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.851434    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5pt2\" (UniqueName: \"kubernetes.io/projected/80304af3-45a3-466b-a4ea-d7ac748811f3-kube-api-access-s5pt2\") pod \"node-resolver-qx4xs\" (UID: \"80304af3-45a3-466b-a4ea-d7ac748811f3\") " pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.852110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.852090    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lv9k5\" (UniqueName: \"kubernetes.io/projected/a9784b34-249f-4f94-9ffb-307a2cece9ea-kube-api-access-lv9k5\") pod \"aws-ebs-csi-driver-node-fq92x\" (UID: \"a9784b34-249f-4f94-9ffb-307a2cece9ea\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.852416 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.852399    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6wcx\" (UniqueName: \"kubernetes.io/projected/121c696e-d742-4a44-beb3-2caf250a1b96-kube-api-access-c6wcx\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:20.927736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.927701    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:20.936035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.936015    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5pfl5"
Sep 19 12:25:20.945727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.945703    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jpl8l"
Sep 19 12:25:20.950825 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.950808    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7qjb8"
Sep 19 12:25:20.958545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.958527    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cxgd2"
Sep 19 12:25:20.963382 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.963361    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x"
Sep 19 12:25:20.969989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.969971    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-qx4xs"
Sep 19 12:25:20.977649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.977598    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8gsh2"
Sep 19 12:25:20.982119 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:20.982099    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:21.346648 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.346566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:21.346648 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.346604    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:21.346854 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.346712    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:21.346854 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.346729    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:21.346854 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.346781    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret podName:ae5d7b23-07f5-47d3-af44-d1cb64091bca nodeName:}" failed. No retries permitted until 2026-09-19 12:25:22.346766236 +0000 UTC m=+5.707009154 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret") pod "global-pull-secret-syncer-wf7d9" (UID: "ae5d7b23-07f5-47d3-af44-d1cb64091bca") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:21.346854 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.346795    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:22.346789659 +0000 UTC m=+5.707032578 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:21.447419 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.447381    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:21.447595 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.447553    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:21.447595 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.447578    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:21.447595 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.447592    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jpkqb for pod openshift-network-diagnostics/network-check-target-c2nxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:21.447732 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:21.447651    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb podName:2d320379-f1e7-4f73-ba51-e2559ceb1a1e nodeName:}" failed. No retries permitted until 2026-09-19 12:25:22.447637964 +0000 UTC m=+5.807880883 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jpkqb" (UniqueName: "kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb") pod "network-check-target-c2nxz" (UID: "2d320379-f1e7-4f73-ba51-e2559ceb1a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:21.558763 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.558549    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80304af3_45a3_466b_a4ea_d7ac748811f3.slice/crio-26896b7aa29424bdbd11617ac9f21ce485a722798c39faae2e51a2aa9463217d WatchSource:0}: Error finding container 26896b7aa29424bdbd11617ac9f21ce485a722798c39faae2e51a2aa9463217d: Status 404 returned error can't find the container with id 26896b7aa29424bdbd11617ac9f21ce485a722798c39faae2e51a2aa9463217d
Sep 19 12:25:21.559951 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.559846    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc00d1258_add9_4b09_a9a6_4d6c82f01ec1.slice/crio-5701b797c365506e01c097a0d2e374fae31d5a77c5b7015cc71802b7a785bf77 WatchSource:0}: Error finding container 5701b797c365506e01c097a0d2e374fae31d5a77c5b7015cc71802b7a785bf77: Status 404 returned error can't find the container with id 5701b797c365506e01c097a0d2e374fae31d5a77c5b7015cc71802b7a785bf77
Sep 19 12:25:21.563918 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.563897    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e407c77_d390_4e67_ae03_f083fa4acf52.slice/crio-77ca6c65d5d756a1521fa5d37fd65ace85bb3fc78845f7218bc21350fd2462aa WatchSource:0}: Error finding container 77ca6c65d5d756a1521fa5d37fd65ace85bb3fc78845f7218bc21350fd2462aa: Status 404 returned error can't find the container with id 77ca6c65d5d756a1521fa5d37fd65ace85bb3fc78845f7218bc21350fd2462aa
Sep 19 12:25:21.565015 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.564993    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fb3f992_7bca_4383_89a9_97a0eaa45a54.slice/crio-09ae2ab56dbf99dd4424e8521e8323b4933fa0bd8f5cd0f8a0ecaac00aa8fdf0 WatchSource:0}: Error finding container 09ae2ab56dbf99dd4424e8521e8323b4933fa0bd8f5cd0f8a0ecaac00aa8fdf0: Status 404 returned error can't find the container with id 09ae2ab56dbf99dd4424e8521e8323b4933fa0bd8f5cd0f8a0ecaac00aa8fdf0
Sep 19 12:25:21.566660 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.566633    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2291616_999a_4193_8882_f43d2174c60e.slice/crio-277826fe8332f34108488d65096a7c0acbf6d30bbbe356506c64b0ba8d2c6c28 WatchSource:0}: Error finding container 277826fe8332f34108488d65096a7c0acbf6d30bbbe356506c64b0ba8d2c6c28: Status 404 returned error can't find the container with id 277826fe8332f34108488d65096a7c0acbf6d30bbbe356506c64b0ba8d2c6c28
Sep 19 12:25:21.566845 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.566829    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0eb4745_b388_4c27_aea0_da13c0976513.slice/crio-e29bc235d370a36d93a594ed321e7c46556e4222511c50f29b1de1985945bc4e WatchSource:0}: Error finding container e29bc235d370a36d93a594ed321e7c46556e4222511c50f29b1de1985945bc4e: Status 404 returned error can't find the container with id e29bc235d370a36d93a594ed321e7c46556e4222511c50f29b1de1985945bc4e
Sep 19 12:25:21.568237 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.568216    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e9fe70_c0cb_4715_82d6_4e4d529484c7.slice/crio-00a989a6a87f100511cd6087627e7f30db016638618c11ef30ffb885e721b8f6 WatchSource:0}: Error finding container 00a989a6a87f100511cd6087627e7f30db016638618c11ef30ffb885e721b8f6: Status 404 returned error can't find the container with id 00a989a6a87f100511cd6087627e7f30db016638618c11ef30ffb885e721b8f6
Sep 19 12:25:21.569010 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.568952    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9784b34_249f_4f94_9ffb_307a2cece9ea.slice/crio-ea823a5c5519013dd85336cb0a316c290e715fa6270bda5cc5db35ae0cbd74d3 WatchSource:0}: Error finding container ea823a5c5519013dd85336cb0a316c290e715fa6270bda5cc5db35ae0cbd74d3: Status 404 returned error can't find the container with id ea823a5c5519013dd85336cb0a316c290e715fa6270bda5cc5db35ae0cbd74d3
Sep 19 12:25:21.570870 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:21.570482    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638fdc50_3117_472b_a7f7_133a1c8d015a.slice/crio-e6a9db4edb943080b635615d5c33b89c65e4d44f4c22f5182569d1dc67074ce3 WatchSource:0}: Error finding container e6a9db4edb943080b635615d5c33b89c65e4d44f4c22f5182569d1dc67074ce3: Status 404 returned error can't find the container with id e6a9db4edb943080b635615d5c33b89c65e4d44f4c22f5182569d1dc67074ce3
Sep 19 12:25:21.669170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.669140    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 12:20:20 +0000 UTC" deadline="2028-04-30 12:10:48.97957637 +0000 UTC"
Sep 19 12:25:21.669170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.669167    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14135h45m27.310412226s"
Sep 19 12:25:21.744123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.744089    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerStarted","Data":"e6a9db4edb943080b635615d5c33b89c65e4d44f4c22f5182569d1dc67074ce3"}
Sep 19 12:25:21.745434 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.745403    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x" event={"ID":"a9784b34-249f-4f94-9ffb-307a2cece9ea","Type":"ContainerStarted","Data":"ea823a5c5519013dd85336cb0a316c290e715fa6270bda5cc5db35ae0cbd74d3"}
Sep 19 12:25:21.746625 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.746599    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jpl8l" event={"ID":"e2291616-999a-4193-8882-f43d2174c60e","Type":"ContainerStarted","Data":"277826fe8332f34108488d65096a7c0acbf6d30bbbe356506c64b0ba8d2c6c28"}
Sep 19 12:25:21.747670 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.747641    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jr2dg" event={"ID":"0e407c77-d390-4e67-ae03-f083fa4acf52","Type":"ContainerStarted","Data":"77ca6c65d5d756a1521fa5d37fd65ace85bb3fc78845f7218bc21350fd2462aa"}
Sep 19 12:25:21.748699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.748681    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cxgd2" event={"ID":"c00d1258-add9-4b09-a9a6-4d6c82f01ec1","Type":"ContainerStarted","Data":"5701b797c365506e01c097a0d2e374fae31d5a77c5b7015cc71802b7a785bf77"}
Sep 19 12:25:21.749934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.749898    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qx4xs" event={"ID":"80304af3-45a3-466b-a4ea-d7ac748811f3","Type":"ContainerStarted","Data":"26896b7aa29424bdbd11617ac9f21ce485a722798c39faae2e51a2aa9463217d"}
Sep 19 12:25:21.751903 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.751883    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal" event={"ID":"769ded29dc5f76d39a7c0031f5b9ed81","Type":"ContainerStarted","Data":"002054131cbd7b917d4d027625810fba18dcdee0ef608db016691de7e4358cbe"}
Sep 19 12:25:21.753082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.753057    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8gsh2" event={"ID":"a7e9fe70-c0cb-4715-82d6-4e4d529484c7","Type":"ContainerStarted","Data":"00a989a6a87f100511cd6087627e7f30db016638618c11ef30ffb885e721b8f6"}
Sep 19 12:25:21.754230 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.754208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5pfl5" event={"ID":"b0eb4745-b388-4c27-aea0-da13c0976513","Type":"ContainerStarted","Data":"e29bc235d370a36d93a594ed321e7c46556e4222511c50f29b1de1985945bc4e"}
Sep 19 12:25:21.755384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.755366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"09ae2ab56dbf99dd4424e8521e8323b4933fa0bd8f5cd0f8a0ecaac00aa8fdf0"}
Sep 19 12:25:21.766616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:21.766581    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-60.ec2.internal" podStartSLOduration=2.766569208 podStartE2EDuration="2.766569208s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:21.766526047 +0000 UTC m=+5.126768988" watchObservedRunningTime="2026-09-19 12:25:21.766569208 +0000 UTC m=+5.126812149"
Sep 19 12:25:22.355120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.355080    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:22.355297 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.355140    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:22.355297 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.355265    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:22.355427 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.355331    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret podName:ae5d7b23-07f5-47d3-af44-d1cb64091bca nodeName:}" failed. No retries permitted until 2026-09-19 12:25:24.355311737 +0000 UTC m=+7.715554672 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret") pod "global-pull-secret-syncer-wf7d9" (UID: "ae5d7b23-07f5-47d3-af44-d1cb64091bca") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:22.355427 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.355398    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:22.355549 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.355431    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:24.355418722 +0000 UTC m=+7.715661658 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:22.456164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.456124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:22.456366 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.456349    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:22.456441 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.456373    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:22.456441 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.456387    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jpkqb for pod openshift-network-diagnostics/network-check-target-c2nxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:22.456576 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.456443    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb podName:2d320379-f1e7-4f73-ba51-e2559ceb1a1e nodeName:}" failed. No retries permitted until 2026-09-19 12:25:24.456427479 +0000 UTC m=+7.816670414 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jpkqb" (UniqueName: "kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb") pod "network-check-target-c2nxz" (UID: "2d320379-f1e7-4f73-ba51-e2559ceb1a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:22.670121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.669984    2571 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-18 12:20:20 +0000 UTC" deadline="2028-05-31 23:27:08.112182324 +0000 UTC"
Sep 19 12:25:22.670121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.670045    2571 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14891h1m45.442141817s"
Sep 19 12:25:22.735786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.735004    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:22.735786 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.735135    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:22.735786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.735543    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:22.735786 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.735622    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:22.735786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.735670    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:22.735786 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:22.735755    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:22.780270 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.779694    2571 generic.go:356] "Generic (PLEG): container finished" podID="794c2de5d33bcf417220a257d18a1d6b" containerID="1ffa9221ecfcb4bb2821e26b4e23c4a0beeaee1917c64ae775decf1faf4e7435" exitCode=0
Sep 19 12:25:22.780270 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:22.779801    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal" event={"ID":"794c2de5d33bcf417220a257d18a1d6b","Type":"ContainerDied","Data":"1ffa9221ecfcb4bb2821e26b4e23c4a0beeaee1917c64ae775decf1faf4e7435"}
Sep 19 12:25:23.809744 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:23.809010    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal" event={"ID":"794c2de5d33bcf417220a257d18a1d6b","Type":"ContainerStarted","Data":"9c9aaa4ad407bacff4676478a295fe5a0033b71b4947fc646c7afbd277db9988"}
Sep 19 12:25:23.825780 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:23.824885    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-60.ec2.internal" podStartSLOduration=4.824864996 podStartE2EDuration="4.824864996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:23.824536744 +0000 UTC m=+7.184779685" watchObservedRunningTime="2026-09-19 12:25:23.824864996 +0000 UTC m=+7.185107933"
Sep 19 12:25:24.371096 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:24.371058    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:24.371280 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:24.371151    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:24.371280 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.371275    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:24.371387 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.371334    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:28.371315622 +0000 UTC m=+11.731558557 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:24.371479 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.371396    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:24.371479 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.371429    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret podName:ae5d7b23-07f5-47d3-af44-d1cb64091bca nodeName:}" failed. No retries permitted until 2026-09-19 12:25:28.371418175 +0000 UTC m=+11.731661094 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret") pod "global-pull-secret-syncer-wf7d9" (UID: "ae5d7b23-07f5-47d3-af44-d1cb64091bca") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:24.472402 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:24.471778    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:24.472402 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.471988    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:24.472402 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.472006    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:24.472402 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.472019    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jpkqb for pod openshift-network-diagnostics/network-check-target-c2nxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:24.472402 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.472074    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb podName:2d320379-f1e7-4f73-ba51-e2559ceb1a1e nodeName:}" failed. No retries permitted until 2026-09-19 12:25:28.472056342 +0000 UTC m=+11.832299280 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jpkqb" (UniqueName: "kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb") pod "network-check-target-c2nxz" (UID: "2d320379-f1e7-4f73-ba51-e2559ceb1a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:24.733784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:24.733653    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:24.733784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:24.733691    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:24.733784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:24.733657    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:24.734089 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.733790    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:24.734089 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.733880    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:24.734089 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:24.733947    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:26.737800 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:26.737764    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:26.738216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:26.734693    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:26.738216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:26.738005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:26.738216 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:26.738001    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:26.738958 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:26.738905    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:26.739045 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:26.738970    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:28.405852 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:28.405347    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:28.405852 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:28.405414    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:28.405852 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.405599    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:28.405852 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.405660    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret podName:ae5d7b23-07f5-47d3-af44-d1cb64091bca nodeName:}" failed. No retries permitted until 2026-09-19 12:25:36.405641594 +0000 UTC m=+19.765884532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret") pod "global-pull-secret-syncer-wf7d9" (UID: "ae5d7b23-07f5-47d3-af44-d1cb64091bca") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:28.405852 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.405765    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:28.405852 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.405810    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:36.405796486 +0000 UTC m=+19.766039410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:28.506508 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:28.506473    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:28.506668 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.506631    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:28.506668 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.506648    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:28.506668 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.506659    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jpkqb for pod openshift-network-diagnostics/network-check-target-c2nxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:28.506793 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.506715    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb podName:2d320379-f1e7-4f73-ba51-e2559ceb1a1e nodeName:}" failed. No retries permitted until 2026-09-19 12:25:36.506698588 +0000 UTC m=+19.866941520 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jpkqb" (UniqueName: "kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb") pod "network-check-target-c2nxz" (UID: "2d320379-f1e7-4f73-ba51-e2559ceb1a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:28.735613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:28.734358    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:28.735613 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.734474    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:28.735613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:28.734812    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:28.735613 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.734908    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:28.735613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:28.734958    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:28.735613 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:28.735022    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:30.734352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:30.734320    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:30.734352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:30.734349    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:30.734783 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:30.734326    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:30.734783 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:30.734441    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:30.734783 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:30.734529    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:30.734783 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:30.734645    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:32.733631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:32.733567    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:32.733631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:32.733610    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:32.734127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:32.733565    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:32.734127 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:32.733706    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:32.734127 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:32.733793    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:32.734127 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:32.733857    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:34.733392 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:34.733363    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:34.733848 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:34.733362    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:34.733848 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:34.733499    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:34.733848 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:34.733493    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:34.733848 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:34.733567    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:34.733848 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:34.733670    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:36.458662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:36.458622    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:36.459112 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:36.458716    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:36.459112 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.458778    2571 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:36.459112 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.458795    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:36.459112 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.458842    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:52.458826855 +0000 UTC m=+35.819069792 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 19 12:25:36.459112 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.458860    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret podName:ae5d7b23-07f5-47d3-af44-d1cb64091bca nodeName:}" failed. No retries permitted until 2026-09-19 12:25:52.458851372 +0000 UTC m=+35.819094296 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret") pod "global-pull-secret-syncer-wf7d9" (UID: "ae5d7b23-07f5-47d3-af44-d1cb64091bca") : object "kube-system"/"original-pull-secret" not registered
Sep 19 12:25:36.559335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:36.559302    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:36.559529 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.559508    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 19 12:25:36.559604 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.559535    2571 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 19 12:25:36.559604 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.559549    2571 projected.go:196] Error preparing data for projected volume kube-api-access-jpkqb for pod openshift-network-diagnostics/network-check-target-c2nxz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:36.559708 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.559616    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb podName:2d320379-f1e7-4f73-ba51-e2559ceb1a1e nodeName:}" failed. No retries permitted until 2026-09-19 12:25:52.559598625 +0000 UTC m=+35.919841550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jpkqb" (UniqueName: "kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb") pod "network-check-target-c2nxz" (UID: "2d320379-f1e7-4f73-ba51-e2559ceb1a1e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 19 12:25:36.733554 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:36.733470    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:36.733554 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:36.733512    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:36.733746 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:36.733476    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:36.733746 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.733597    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:36.733746 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.733692    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:36.733893 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:36.733750    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:38.734400 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:38.734372    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:38.734836 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:38.734480    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:38.734836 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:38.734507    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:38.734836 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:38.734603    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:38.734836 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:38.734637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:38.734836 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:38.734684    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:39.839345 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.838965    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-qx4xs" event={"ID":"80304af3-45a3-466b-a4ea-d7ac748811f3","Type":"ContainerStarted","Data":"147d3d94fa05849d691cd377b93fd7016d9c8bae2d65767fa66add7a8393d747"}
Sep 19 12:25:39.840967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.840938    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8gsh2" event={"ID":"a7e9fe70-c0cb-4715-82d6-4e4d529484c7","Type":"ContainerStarted","Data":"ca4a246ac65a8f9720f17c6f6cb7a60de77983c876a23f922b943aa613950472"}
Sep 19 12:25:39.844782 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.844761    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"d907e1164744c34b1a7593d90f7c607dbb79b4342388f2990cd02cfb83eabc0e"}
Sep 19 12:25:39.844904 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.844834    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"08dbceb078dacf00558643bbb679e9054abae42115df88b3d8dc47d4d33a5f5d"}
Sep 19 12:25:39.844904 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.844887    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"1f01fb5017ee8943200de03114b253176e9bf0f2656272eddf62d9f8def82a56"}
Sep 19 12:25:39.844987 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.844935    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"299449b788e77bbb58a28eeb8b24a3256d5ceb61cd96ea8779e4875d4ddf691b"}
Sep 19 12:25:39.844987 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.844981    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"e32bafb2dd4b618478bfdf2ba3d4d099c8c0971cbe26a2b5b79b8820107ffa57"}
Sep 19 12:25:39.845064 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.845010    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"aa4cb967602a49b655b48f07bfb57fd98127b10ed51283407398557c4c063a57"}
Sep 19 12:25:39.846350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.846327    2571 generic.go:356] "Generic (PLEG): container finished" podID="638fdc50-3117-472b-a7f7-133a1c8d015a" containerID="f64adfc4ff7fdb825a14768d4f50d5ebeeb647188db68405f3ac645dc7fc9a08" exitCode=0
Sep 19 12:25:39.846446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.846396    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerDied","Data":"f64adfc4ff7fdb825a14768d4f50d5ebeeb647188db68405f3ac645dc7fc9a08"}
Sep 19 12:25:39.848053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.848032    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x" event={"ID":"a9784b34-249f-4f94-9ffb-307a2cece9ea","Type":"ContainerStarted","Data":"96f2d36069fa967bdf4ec0e1f8462ef918102ecae886fbafa43b6b3f42962842"}
Sep 19 12:25:39.849585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.849561    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jpl8l" event={"ID":"e2291616-999a-4193-8882-f43d2174c60e","Type":"ContainerStarted","Data":"2c4f87592d53806a225a26517cc1bbfec7b758ae3fc6bd196f9bb7a82f5b3095"}
Sep 19 12:25:39.850879 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.850861    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jr2dg" event={"ID":"0e407c77-d390-4e67-ae03-f083fa4acf52","Type":"ContainerStarted","Data":"1c032cbf2b1fe602e6730f3738d1485c34ef3fb74a6b77bd8df4b0d572111383"}
Sep 19 12:25:39.852244 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.852221    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cxgd2" event={"ID":"c00d1258-add9-4b09-a9a6-4d6c82f01ec1","Type":"ContainerStarted","Data":"191f41be180063176de8794d9a0b5682d7b0d0041968292c0d1e1dbb7f18260c"}
Sep 19 12:25:39.853625 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.853589    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-qx4xs" podStartSLOduration=4.503565196 podStartE2EDuration="21.853576622s" totalImagesPullingTime="17.350011426s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:39.853078526 +0000 UTC m=+23.213321472" watchObservedRunningTime="2026-09-19 12:25:39.853576622 +0000 UTC m=+23.213819563"
Sep 19 12:25:39.888222 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.888185    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cxgd2" podStartSLOduration=4.494527738 podStartE2EDuration="21.888170716s" totalImagesPullingTime="17.393642978s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:39.887813181 +0000 UTC m=+23.248056119" watchObservedRunningTime="2026-09-19 12:25:39.888170716 +0000 UTC m=+23.248413657"
Sep 19 12:25:39.902707 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.902666    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jr2dg" podStartSLOduration=11.995985417 podStartE2EDuration="21.902646849s" totalImagesPullingTime="9.906661432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:39.902314298 +0000 UTC m=+23.262557240" watchObservedRunningTime="2026-09-19 12:25:39.902646849 +0000 UTC m=+23.262889789"
Sep 19 12:25:39.927508 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.927453    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jpl8l" podStartSLOduration=4.5868044690000005 podStartE2EDuration="21.927438309s" totalImagesPullingTime="17.34063384s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:39.927075602 +0000 UTC m=+23.287318543" watchObservedRunningTime="2026-09-19 12:25:39.927438309 +0000 UTC m=+23.287681252"
Sep 19 12:25:39.970258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:39.970237    2571 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 19 12:25:40.368910 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.368811    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:40.369688 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.369536    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:40.389434 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.389383    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8gsh2" podStartSLOduration=5.048353759 podStartE2EDuration="22.389361171s" totalImagesPullingTime="17.341007412s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:39.949534168 +0000 UTC m=+23.309777109" watchObservedRunningTime="2026-09-19 12:25:40.389361171 +0000 UTC m=+23.749604113"
Sep 19 12:25:40.704424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.704264    2571 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-19T12:25:39.970255591Z","UUID":"d5e85878-92bc-4336-8024-cd301b7afc5f","Handler":null,"Name":"","Endpoint":""}
Sep 19 12:25:40.706244 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.706118    2571 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 19 12:25:40.706405 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.706156    2571 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 19 12:25:40.734249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.734193    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:40.734396 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.734310    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:40.734396 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:40.734321    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:40.734546 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:40.734475    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:40.734546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.734512    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:40.734652 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:40.734581    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:40.856892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.856362    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5pfl5" event={"ID":"b0eb4745-b388-4c27-aea0-da13c0976513","Type":"ContainerStarted","Data":"49a3d62adfa0835d5eb6fb0f01f695a63aa649a4f96f94c82aeb30fa3db23049"}
Sep 19 12:25:40.859064 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.859037    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x" event={"ID":"a9784b34-249f-4f94-9ffb-307a2cece9ea","Type":"ContainerStarted","Data":"0e0b7977c88725169d6da9d2160a06ba0dd2febd33111d27a58349d17b64e2cc"}
Sep 19 12:25:40.859990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.859970    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:40.860069 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.860027    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jr2dg"
Sep 19 12:25:40.874093 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:40.874053    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5pfl5" podStartSLOduration=5.533828981 podStartE2EDuration="22.87403816s" totalImagesPullingTime="17.340209179s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:40.87383377 +0000 UTC m=+24.234076710" watchObservedRunningTime="2026-09-19 12:25:40.87403816 +0000 UTC m=+24.234281138"
Sep 19 12:25:41.018023 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.017915    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:41.103754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.103718    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:41.295736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.295660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:41.359169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.359136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:41.863776 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.863683    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x" event={"ID":"a9784b34-249f-4f94-9ffb-307a2cece9ea","Type":"ContainerStarted","Data":"29dc35fc55847dc5e9042e28a2432ae403bdfe15d41d17b29302bb13212f218b"}
Sep 19 12:25:41.867738 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.867672    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"b545d6e465d299d6c64d3a3a5a13415023512b374adb6593325d94842c61dcd5"}
Sep 19 12:25:41.881937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:41.881892    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fq92x" podStartSLOduration=5.017375377 podStartE2EDuration="23.881878114s" totalImagesPullingTime="18.864502737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:41.881807946 +0000 UTC m=+25.242050947" watchObservedRunningTime="2026-09-19 12:25:41.881878114 +0000 UTC m=+25.242121056"
Sep 19 12:25:42.337072 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:42.336984    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:42.437961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:42.437918    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:42.734547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:42.734451    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:42.734699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:42.734451    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:42.734699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:42.734600    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:42.734699 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:42.734598    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:42.734836 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:42.734693    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:42.734836 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:42.734762    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:42.916419 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:42.916383    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:43.126240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.126159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:43.224837 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.224797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:43.508833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.508737    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:43.622261 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.622236    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:43.877780 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.877497    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" event={"ID":"5fb3f992-7bca-4383-89a9-97a0eaa45a54","Type":"ContainerStarted","Data":"610dff2b820475a431e39e385acdba6fef9f3a5c347d154dd5cb733207fe6b9e"}
Sep 19 12:25:43.878247 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.877857    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:43.878409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.878386    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:43.878499 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.878493    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:43.885268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.885243    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerStarted","Data":"96bc591c52852352b127ba8ed5783da1abd39255230e7314352f2967d452a1d0"}
Sep 19 12:25:43.896035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.896009    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:43.896139 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.896114    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:25:43.915756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:43.915699    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5" podStartSLOduration=8.556110473 podStartE2EDuration="25.915684846s" totalImagesPullingTime="17.359574373s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:43.915187693 +0000 UTC m=+27.275430635" watchObservedRunningTime="2026-09-19 12:25:43.915684846 +0000 UTC m=+27.275927786"
Sep 19 12:25:44.733971 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:44.733938    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:44.733971 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:44.733971    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:44.734708 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:44.733938    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:44.734708 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:44.734047    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:44.734708 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:44.734109    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:44.734708 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:44.734182    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:44.890609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:44.890570    2571 generic.go:356] "Generic (PLEG): container finished" podID="638fdc50-3117-472b-a7f7-133a1c8d015a" containerID="96bc591c52852352b127ba8ed5783da1abd39255230e7314352f2967d452a1d0" exitCode=0
Sep 19 12:25:44.890775 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:44.890667    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerDied","Data":"96bc591c52852352b127ba8ed5783da1abd39255230e7314352f2967d452a1d0"}
Sep 19 12:25:45.594648 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.594614    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4569q"]
Sep 19 12:25:45.594795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.594768    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:45.594926 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:45.594888    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:45.598937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.598791    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c2nxz"]
Sep 19 12:25:45.598937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.598904    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:45.599101 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:45.599005    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:45.599688 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.599666    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wf7d9"]
Sep 19 12:25:45.599780 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.599748    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:45.599862 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:45.599839    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:45.962141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:45.962056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:46.062682 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:46.062655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:46.715597 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:46.715373    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:46.815831 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:46.815797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:46.896421 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:46.896383    2571 generic.go:356] "Generic (PLEG): container finished" podID="638fdc50-3117-472b-a7f7-133a1c8d015a" containerID="b9d2334ab146b0745b586c089abc616cada9fa7f367f3d5a3d83ab7a1c978e02" exitCode=0
Sep 19 12:25:46.896583 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:46.896454    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerDied","Data":"b9d2334ab146b0745b586c089abc616cada9fa7f367f3d5a3d83ab7a1c978e02"}
Sep 19 12:25:47.734376 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:47.734347    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:47.734780 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:47.734347    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:47.734780 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:47.734496    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:47.734780 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:47.734354    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:47.734780 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:47.734585    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:47.734780 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:47.734632    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:48.903393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:48.903359    2571 generic.go:356] "Generic (PLEG): container finished" podID="638fdc50-3117-472b-a7f7-133a1c8d015a" containerID="f4d66c13182b760300c85ef0f090f6fa46fdaa6a3c7cbdf043dc48f465516d8d" exitCode=0
Sep 19 12:25:48.904147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:48.903418    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerDied","Data":"f4d66c13182b760300c85ef0f090f6fa46fdaa6a3c7cbdf043dc48f465516d8d"}
Sep 19 12:25:49.734439 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:49.734357    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:49.734439 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:49.734383    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:49.734712 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:49.734534    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4569q" podUID="121c696e-d742-4a44-beb3-2caf250a1b96"
Sep 19 12:25:49.734712 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:49.734597    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-c2nxz" podUID="2d320379-f1e7-4f73-ba51-e2559ceb1a1e"
Sep 19 12:25:49.734712 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:49.734650    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:49.734859 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:49.734748    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wf7d9" podUID="ae5d7b23-07f5-47d3-af44-d1cb64091bca"
Sep 19 12:25:51.155338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.155248    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:51.250183 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.250143    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:51.641819 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.641785    2571 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-60.ec2.internal" event="NodeReady"
Sep 19 12:25:51.641996 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.641939    2571 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 19 12:25:51.694347 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.694269    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-c4bvb"]
Sep 19 12:25:51.727910 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.727873    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s882b"]
Sep 19 12:25:51.728069 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.727980    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.730997 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.730970    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 19 12:25:51.731236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.731219    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 19 12:25:51.731325 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.731272    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6jvqq\""
Sep 19 12:25:51.742232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.742213    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4bvb"]
Sep 19 12:25:51.742326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.742299    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s882b"]
Sep 19 12:25:51.742326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.742312    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.742429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.742375    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:51.742429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.742386    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:51.742562 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.742548    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:51.745005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.744973    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 19 12:25:51.745123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.745020    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 19 12:25:51.745123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.745090    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 19 12:25:51.745123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.744984    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kdhkj\""
Sep 19 12:25:51.745279 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.744984    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g8nvf\""
Sep 19 12:25:51.745472 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.745441    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 19 12:25:51.745560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.745498    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 19 12:25:51.745560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.745533    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 19 12:25:51.745560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.745544    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 19 12:25:51.747680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.747662    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n4klm\""
Sep 19 12:25:51.871432 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.871399    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccd17cda-e0b9-44c7-9375-7769b1a38a39-cert\") pod \"ingress-canary-s882b\" (UID: \"ccd17cda-e0b9-44c7-9375-7769b1a38a39\") " pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.871628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.871453    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e90124-26b9-40ad-9281-d14a5bcc74e4-config-volume\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.871628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.871493    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6pgk\" (UniqueName: \"kubernetes.io/projected/b2e90124-26b9-40ad-9281-d14a5bcc74e4-kube-api-access-p6pgk\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.871628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.871599    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e90124-26b9-40ad-9281-d14a5bcc74e4-tmp-dir\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.871839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.871637    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j7mm9\" (UniqueName: \"kubernetes.io/projected/ccd17cda-e0b9-44c7-9375-7769b1a38a39-kube-api-access-j7mm9\") pod \"ingress-canary-s882b\" (UID: \"ccd17cda-e0b9-44c7-9375-7769b1a38a39\") " pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.871839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.871709    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.972938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.972905    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e90124-26b9-40ad-9281-d14a5bcc74e4-tmp-dir\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.972938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.972942    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j7mm9\" (UniqueName: \"kubernetes.io/projected/ccd17cda-e0b9-44c7-9375-7769b1a38a39-kube-api-access-j7mm9\") pod \"ingress-canary-s882b\" (UID: \"ccd17cda-e0b9-44c7-9375-7769b1a38a39\") " pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.973158 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.973070    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.973158 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.973152    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccd17cda-e0b9-44c7-9375-7769b1a38a39-cert\") pod \"ingress-canary-s882b\" (UID: \"ccd17cda-e0b9-44c7-9375-7769b1a38a39\") " pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.973267 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.973191    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e90124-26b9-40ad-9281-d14a5bcc74e4-config-volume\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.973267 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.973216    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6pgk\" (UniqueName: \"kubernetes.io/projected/b2e90124-26b9-40ad-9281-d14a5bcc74e4-kube-api-access-p6pgk\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.973370 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:51.973273    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:51.973370 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:51.973343    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls podName:b2e90124-26b9-40ad-9281-d14a5bcc74e4 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:52.473325075 +0000 UTC m=+35.833568006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls") pod "dns-default-c4bvb" (UID: "b2e90124-26b9-40ad-9281-d14a5bcc74e4") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:51.977810 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.977624    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccd17cda-e0b9-44c7-9375-7769b1a38a39-cert\") pod \"ingress-canary-s882b\" (UID: \"ccd17cda-e0b9-44c7-9375-7769b1a38a39\") " pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.983431 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.983407    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b2e90124-26b9-40ad-9281-d14a5bcc74e4-tmp-dir\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.983761 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.983740    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b2e90124-26b9-40ad-9281-d14a5bcc74e4-config-volume\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:51.985192 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.985168    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j7mm9\" (UniqueName: \"kubernetes.io/projected/ccd17cda-e0b9-44c7-9375-7769b1a38a39-kube-api-access-j7mm9\") pod \"ingress-canary-s882b\" (UID: \"ccd17cda-e0b9-44c7-9375-7769b1a38a39\") " pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:51.985494 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:51.985455    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6pgk\" (UniqueName: \"kubernetes.io/projected/b2e90124-26b9-40ad-9281-d14a5bcc74e4-kube-api-access-p6pgk\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:52.053682 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.053644    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s882b"
Sep 19 12:25:52.203011 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.202977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:52.263942 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.263817    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:52.477872 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.477841    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:25:52.477872 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.477874    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:52.478085 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.477903    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:52.478085 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.478013    2571 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 19 12:25:52.478184 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.478091    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs podName:121c696e-d742-4a44-beb3-2caf250a1b96 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:24.478068785 +0000 UTC m=+67.838311727 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs") pod "network-metrics-daemon-4569q" (UID: "121c696e-d742-4a44-beb3-2caf250a1b96") : secret "metrics-daemon-secret" not found
Sep 19 12:25:52.478184 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.478013    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:52.478294 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.478195    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls podName:b2e90124-26b9-40ad-9281-d14a5bcc74e4 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:53.478176116 +0000 UTC m=+36.838419049 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls") pod "dns-default-c4bvb" (UID: "b2e90124-26b9-40ad-9281-d14a5bcc74e4") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:52.480559 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.480536    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae5d7b23-07f5-47d3-af44-d1cb64091bca-original-pull-secret\") pod \"global-pull-secret-syncer-wf7d9\" (UID: \"ae5d7b23-07f5-47d3-af44-d1cb64091bca\") " pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:52.578932 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.578841    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:52.581726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.581700    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpkqb\" (UniqueName: \"kubernetes.io/projected/2d320379-f1e7-4f73-ba51-e2559ceb1a1e-kube-api-access-jpkqb\") pod \"network-check-target-c2nxz\" (UID: \"2d320379-f1e7-4f73-ba51-e2559ceb1a1e\") " pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:52.623352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.623320    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qnpdg"]
Sep 19 12:25:52.633105 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.633075    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:52.651287 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.651259    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qnpdg"]
Sep 19 12:25:52.651405 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.651387    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.653985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.653961    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 19 12:25:52.653985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.653979    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-765zc\""
Sep 19 12:25:52.654445 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.654422    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 19 12:25:52.655065 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.655045    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 19 12:25:52.655176 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.655071    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 19 12:25:52.666255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.666234    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:25:52.671393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.671375    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wf7d9"
Sep 19 12:25:52.752801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.752765    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:52.780635 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.780602    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.780803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.780674    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29a6e79e-a793-4ed7-825c-284e062e97fc-crio-socket\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.780803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.780700    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.780803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.780722    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbgdd\" (UniqueName: \"kubernetes.io/projected/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-api-access-cbgdd\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.780803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.780770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29a6e79e-a793-4ed7-825c-284e062e97fc-data-volume\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.793749 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.793724    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"]
Sep 19 12:25:52.814364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.814316    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"
Sep 19 12:25:52.815289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.815270    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"]
Sep 19 12:25:52.820710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.820689    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7f4xm\""
Sep 19 12:25:52.820870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.820849    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 19 12:25:52.820964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.820917    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 19 12:25:52.881610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.881542    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29a6e79e-a793-4ed7-825c-284e062e97fc-crio-socket\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.881610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.881569    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.881610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.881587    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbgdd\" (UniqueName: \"kubernetes.io/projected/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-api-access-cbgdd\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.881632    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29a6e79e-a793-4ed7-825c-284e062e97fc-data-volume\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.881669    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.881677    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.881732    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/29a6e79e-a793-4ed7-825c-284e062e97fc-crio-socket\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.881753    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:25:53.381733041 +0000 UTC m=+36.741975996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.881783    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:52.881875 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:52.881832    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:25:53.381816276 +0000 UTC m=+36.742059195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:52.882147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.882064    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/29a6e79e-a793-4ed7-825c-284e062e97fc-data-volume\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.910005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.909977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:52.911255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.911235    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbgdd\" (UniqueName: \"kubernetes.io/projected/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-api-access-cbgdd\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:52.981968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.981937    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpk8t\" (UniqueName: \"kubernetes.io/projected/ee553785-32df-49c6-b381-cfe61b9b9153-kube-api-access-cpk8t\") pod \"migrator-76794cbd6c-tkwbm\" (UID: \"ee553785-32df-49c6-b381-cfe61b9b9153\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"
Sep 19 12:25:52.992121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:52.992096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:53.082479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.082431    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpk8t\" (UniqueName: \"kubernetes.io/projected/ee553785-32df-49c6-b381-cfe61b9b9153-kube-api-access-cpk8t\") pod \"migrator-76794cbd6c-tkwbm\" (UID: \"ee553785-32df-49c6-b381-cfe61b9b9153\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"
Sep 19 12:25:53.095986 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.095925    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpk8t\" (UniqueName: \"kubernetes.io/projected/ee553785-32df-49c6-b381-cfe61b9b9153-kube-api-access-cpk8t\") pod \"migrator-76794cbd6c-tkwbm\" (UID: \"ee553785-32df-49c6-b381-cfe61b9b9153\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"
Sep 19 12:25:53.125785 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.125744    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"
Sep 19 12:25:53.307694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.307663    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wf7d9"]
Sep 19 12:25:53.311271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.311194    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s882b"]
Sep 19 12:25:53.312063 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.312012    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-c2nxz"]
Sep 19 12:25:53.313498 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:53.312987    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5d7b23_07f5_47d3_af44_d1cb64091bca.slice/crio-597f4aa7a3f0c186aa2b8c140369f0376c093fd69bc5000bfa95cb1c423fa896 WatchSource:0}: Error finding container 597f4aa7a3f0c186aa2b8c140369f0376c093fd69bc5000bfa95cb1c423fa896: Status 404 returned error can't find the container with id 597f4aa7a3f0c186aa2b8c140369f0376c093fd69bc5000bfa95cb1c423fa896
Sep 19 12:25:53.315586 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:53.315559    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccd17cda_e0b9_44c7_9375_7769b1a38a39.slice/crio-0d3478fe6e2e0193b38b1dfa68742d4dcf89b36d8397f497517f0da8cac2207a WatchSource:0}: Error finding container 0d3478fe6e2e0193b38b1dfa68742d4dcf89b36d8397f497517f0da8cac2207a: Status 404 returned error can't find the container with id 0d3478fe6e2e0193b38b1dfa68742d4dcf89b36d8397f497517f0da8cac2207a
Sep 19 12:25:53.316333 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:53.316311    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d320379_f1e7_4f73_ba51_e2559ceb1a1e.slice/crio-513938560bbca845ae25bf30e7317d28164bd7f952046e19ec0a146ada29d9e6 WatchSource:0}: Error finding container 513938560bbca845ae25bf30e7317d28164bd7f952046e19ec0a146ada29d9e6: Status 404 returned error can't find the container with id 513938560bbca845ae25bf30e7317d28164bd7f952046e19ec0a146ada29d9e6
Sep 19 12:25:53.384508 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.384471    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:53.384638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.384538    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:53.384687 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:53.384634    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:53.384687 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:53.384666    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:53.384756 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:53.384702    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:25:54.384682984 +0000 UTC m=+37.744925923 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:53.384756 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:53.384716    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:25:54.384709816 +0000 UTC m=+37.744952738 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:53.399747 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.399722    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm"]
Sep 19 12:25:53.403809 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:53.403783    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee553785_32df_49c6_b381_cfe61b9b9153.slice/crio-52be583f767cc3a109a067451d0693b95caebc21d8dcd68502368a22f4b6659d WatchSource:0}: Error finding container 52be583f767cc3a109a067451d0693b95caebc21d8dcd68502368a22f4b6659d: Status 404 returned error can't find the container with id 52be583f767cc3a109a067451d0693b95caebc21d8dcd68502368a22f4b6659d
Sep 19 12:25:53.485371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.485345    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:53.485521 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:53.485503    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:53.485587 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:53.485579    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls podName:b2e90124-26b9-40ad-9281-d14a5bcc74e4 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:55.485558961 +0000 UTC m=+38.845801902 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls") pod "dns-default-c4bvb" (UID: "b2e90124-26b9-40ad-9281-d14a5bcc74e4") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:53.917733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.917444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c2nxz" event={"ID":"2d320379-f1e7-4f73-ba51-e2559ceb1a1e","Type":"ContainerStarted","Data":"513938560bbca845ae25bf30e7317d28164bd7f952046e19ec0a146ada29d9e6"}
Sep 19 12:25:53.921179 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.921150    2571 generic.go:356] "Generic (PLEG): container finished" podID="638fdc50-3117-472b-a7f7-133a1c8d015a" containerID="082db4edb2aa5b3af18b50bc2b9b3daaad36b13686c0a90bfe6e8eb41dd44d8f" exitCode=0
Sep 19 12:25:53.921307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.921181    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerDied","Data":"082db4edb2aa5b3af18b50bc2b9b3daaad36b13686c0a90bfe6e8eb41dd44d8f"}
Sep 19 12:25:53.922658 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.922636    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wf7d9" event={"ID":"ae5d7b23-07f5-47d3-af44-d1cb64091bca","Type":"ContainerStarted","Data":"597f4aa7a3f0c186aa2b8c140369f0376c093fd69bc5000bfa95cb1c423fa896"}
Sep 19 12:25:53.924701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.924637    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm" event={"ID":"ee553785-32df-49c6-b381-cfe61b9b9153","Type":"ContainerStarted","Data":"52be583f767cc3a109a067451d0693b95caebc21d8dcd68502368a22f4b6659d"}
Sep 19 12:25:53.925988 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:53.925939    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s882b" event={"ID":"ccd17cda-e0b9-44c7-9375-7769b1a38a39","Type":"ContainerStarted","Data":"0d3478fe6e2e0193b38b1dfa68742d4dcf89b36d8397f497517f0da8cac2207a"}
Sep 19 12:25:54.393879 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.393100    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:54.393879 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.393184    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:54.393879 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:54.393338    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:54.393879 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:54.393403    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:25:56.393374471 +0000 UTC m=+39.753617390 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:54.393879 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:54.393778    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:54.393879 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:54.393827    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:25:56.393815558 +0000 UTC m=+39.754058476 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:54.505555 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.505520    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-wn25z"]
Sep 19 12:25:54.514113 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.514011    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.520351 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.519702    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 19 12:25:54.521548 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.520978    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 19 12:25:54.521548 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.521003    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-g5686\""
Sep 19 12:25:54.521548 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.521275    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 19 12:25:54.525725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.522650    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-wn25z"]
Sep 19 12:25:54.525725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.522748    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 19 12:25:54.553129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.553008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:54.596418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.596377    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45swm\" (UniqueName: \"kubernetes.io/projected/472c9050-1bbf-443e-8055-ecb1a611db1b-kube-api-access-45swm\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.596610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.596519    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/472c9050-1bbf-443e-8055-ecb1a611db1b-signing-cabundle\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.596610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.596571    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/472c9050-1bbf-443e-8055-ecb1a611db1b-signing-key\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.616768 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.616730    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:54.698689 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.697642    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/472c9050-1bbf-443e-8055-ecb1a611db1b-signing-key\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.698689 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.697715    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45swm\" (UniqueName: \"kubernetes.io/projected/472c9050-1bbf-443e-8055-ecb1a611db1b-kube-api-access-45swm\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.698689 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.697793    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/472c9050-1bbf-443e-8055-ecb1a611db1b-signing-cabundle\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.698689 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.698591    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/472c9050-1bbf-443e-8055-ecb1a611db1b-signing-cabundle\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.702817 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.702754    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/472c9050-1bbf-443e-8055-ecb1a611db1b-signing-key\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.707952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.707905    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45swm\" (UniqueName: \"kubernetes.io/projected/472c9050-1bbf-443e-8055-ecb1a611db1b-kube-api-access-45swm\") pod \"service-ca-55f6fc7b86-wn25z\" (UID: \"472c9050-1bbf-443e-8055-ecb1a611db1b\") " pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.785861 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.785819    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qx4xs_80304af3-45a3-466b-a4ea-d7ac748811f3/dns-node-resolver/0.log"
Sep 19 12:25:54.840900 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.840663    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z"
Sep 19 12:25:54.938256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.937334    2571 generic.go:356] "Generic (PLEG): container finished" podID="638fdc50-3117-472b-a7f7-133a1c8d015a" containerID="a7034ad6f5f060286583a27bc4f7f53ff13c5b1aab01b3f3203957767f83d0d3" exitCode=0
Sep 19 12:25:54.938256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:54.937408    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerDied","Data":"a7034ad6f5f060286583a27bc4f7f53ff13c5b1aab01b3f3203957767f83d0d3"}
Sep 19 12:25:55.032748 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.032700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:55.155806 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.155693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:55.361078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.360429    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-55f6fc7b86-wn25z"]
Sep 19 12:25:55.505009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.504917    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:55.505804 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:55.505012    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:55.505804 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:55.505117    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls podName:b2e90124-26b9-40ad-9281-d14a5bcc74e4 nodeName:}" failed. No retries permitted until 2026-09-19 12:25:59.505093456 +0000 UTC m=+42.865336376 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls") pod "dns-default-c4bvb" (UID: "b2e90124-26b9-40ad-9281-d14a5bcc74e4") : secret "dns-default-metrics-tls" not found
Sep 19 12:25:55.587588 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.587513    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jpl8l_e2291616-999a-4193-8882-f43d2174c60e/node-ca/0.log"
Sep 19 12:25:55.615599 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.615536    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:55.944385 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.944274    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" event={"ID":"638fdc50-3117-472b-a7f7-133a1c8d015a","Type":"ContainerStarted","Data":"50489786f552b57ba06bbb4179ca995f68e51a49e2b009c4a6037c643a5b8524"}
Sep 19 12:25:55.971305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:55.971250    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7qjb8" podStartSLOduration=10.231641032 podStartE2EDuration="37.971229419s" totalImagesPullingTime="27.739588387s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:25:55.970063004 +0000 UTC m=+39.330305948" watchObservedRunningTime="2026-09-19 12:25:55.971229419 +0000 UTC m=+39.331472361"
Sep 19 12:25:56.277919 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:25:56.277883    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472c9050_1bbf_443e_8055_ecb1a611db1b.slice/crio-dcdd6a065a371513b42dd5c754f7e4418a765d172fb164ab03ebf9745cb5c7c0 WatchSource:0}: Error finding container dcdd6a065a371513b42dd5c754f7e4418a765d172fb164ab03ebf9745cb5c7c0: Status 404 returned error can't find the container with id dcdd6a065a371513b42dd5c754f7e4418a765d172fb164ab03ebf9745cb5c7c0
Sep 19 12:25:56.414388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:56.414346    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:56.414590 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:56.414508    2571 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 19 12:25:56.414590 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:56.414579    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:26:00.414560693 +0000 UTC m=+43.774803627 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : configmap "kube-rbac-proxy" not found
Sep 19 12:25:56.414717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:56.414505    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:25:56.414717 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:56.414619    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:56.414717 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:56.414680    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:26:00.414664194 +0000 UTC m=+43.774907117 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:25:56.949085 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:56.949045    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z" event={"ID":"472c9050-1bbf-443e-8055-ecb1a611db1b","Type":"ContainerStarted","Data":"dcdd6a065a371513b42dd5c754f7e4418a765d172fb164ab03ebf9745cb5c7c0"}
Sep 19 12:25:57.004108 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:57.004059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:57.323666 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:57.323636    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:58.116404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:58.116365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:58.221663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:58.221634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:25:59.542504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:25:59.542407    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:25:59.542932 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:59.542579    2571 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 19 12:25:59.542932 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:25:59.542657    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls podName:b2e90124-26b9-40ad-9281-d14a5bcc74e4 nodeName:}" failed. No retries permitted until 2026-09-19 12:26:07.542636961 +0000 UTC m=+50.902879881 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls") pod "dns-default-c4bvb" (UID: "b2e90124-26b9-40ad-9281-d14a5bcc74e4") : secret "dns-default-metrics-tls" not found
Sep 19 12:26:00.447132 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.447098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:26:00.447263 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.447164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:26:00.447300 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:26:00.447270    2571 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 19 12:26:00.447334 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:26:00.447326    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls podName:29a6e79e-a793-4ed7-825c-284e062e97fc nodeName:}" failed. No retries permitted until 2026-09-19 12:26:08.447309915 +0000 UTC m=+51.807552847 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qnpdg" (UID: "29a6e79e-a793-4ed7-825c-284e062e97fc") : secret "insights-runtime-extractor-tls" not found
Sep 19 12:26:00.447572 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.447551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/29a6e79e-a793-4ed7-825c-284e062e97fc-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:26:00.964873 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.964833    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-c2nxz" event={"ID":"2d320379-f1e7-4f73-ba51-e2559ceb1a1e","Type":"ContainerStarted","Data":"0f5b173d02487be69525c9f58b9b2a49389a43e7aecc3c979d02ef2ced403265"}
Sep 19 12:26:00.965340 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.964925    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:26:00.966444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.966423    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wf7d9" event={"ID":"ae5d7b23-07f5-47d3-af44-d1cb64091bca","Type":"ContainerStarted","Data":"dc213b43109caef3539d209f8d439efaafd231de31829340c62c526b28f75944"}
Sep 19 12:26:00.968219 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.968193    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm" event={"ID":"ee553785-32df-49c6-b381-cfe61b9b9153","Type":"ContainerStarted","Data":"a3ed3b95c751ebd19765c28da34d800d0c5f73446bea5059114bff188bceb96e"}
Sep 19 12:26:00.968329 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.968258    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm" event={"ID":"ee553785-32df-49c6-b381-cfe61b9b9153","Type":"ContainerStarted","Data":"5043dc981bfd9e2039354877b40fb50e1c91134799d8a62238c7c6d44dd38b24"}
Sep 19 12:26:00.969625 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.969609    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s882b" event={"ID":"ccd17cda-e0b9-44c7-9375-7769b1a38a39","Type":"ContainerStarted","Data":"bd6e077316e0ceea50fbc7aa2919e46f6ff6e9ae4203178b557860fe8ad8491a"}
Sep 19 12:26:00.971102 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.971080    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z" event={"ID":"472c9050-1bbf-443e-8055-ecb1a611db1b","Type":"ContainerStarted","Data":"04b9d00c21b78e22b7b32a7dba8099c1bbdc11485c79f79541242de2cf14fee3"}
Sep 19 12:26:00.982370 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.982328    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-c2nxz" podStartSLOduration=36.415873621 podStartE2EDuration="42.982316616s" totalImagesPullingTime="6.566442995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:00.981097147 +0000 UTC m=+44.341340087" watchObservedRunningTime="2026-09-19 12:26:00.982316616 +0000 UTC m=+44.342559557"
Sep 19 12:26:00.996566 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:00.996528    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s882b" podStartSLOduration=3.429061631 podStartE2EDuration="9.996514257s" totalImagesPullingTime="6.567452626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:00.996088227 +0000 UTC m=+44.356331174" watchObservedRunningTime="2026-09-19 12:26:00.996514257 +0000 UTC m=+44.356757200"
Sep 19 12:26:01.012814 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:01.012779    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-tkwbm" podStartSLOduration=7.124778712 podStartE2EDuration="9.01276501s" totalImagesPullingTime="1.887986298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:01.011873788 +0000 UTC m=+44.372116727" watchObservedRunningTime="2026-09-19 12:26:01.01276501 +0000 UTC m=+44.373007951"
Sep 19 12:26:01.028022 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:01.027987    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-55f6fc7b86-wn25z" podStartSLOduration=3.033810746 podStartE2EDuration="7.027976273s" totalImagesPullingTime="3.994165527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:01.027063443 +0000 UTC m=+44.387306382" watchObservedRunningTime="2026-09-19 12:26:01.027976273 +0000 UTC m=+44.388219204"
Sep 19 12:26:01.043066 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:01.043028    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-wf7d9" podStartSLOduration=36.461405375 podStartE2EDuration="43.043016223s" totalImagesPullingTime="6.581610848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:01.042497874 +0000 UTC m=+44.402740816" watchObservedRunningTime="2026-09-19 12:26:01.043016223 +0000 UTC m=+44.403259164"
Sep 19 12:26:02.119521 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:02.119483    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:02.227628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:02.227593    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:02.324410 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:02.324375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:02.425133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:02.425068    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:02.524907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:02.524877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:02.625075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:02.625024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:03.011385 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:03.011352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:03.409881 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:03.409800    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:04.212733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:04.212701    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:04.410399 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:04.410368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:04.811073 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:04.811046    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:05.012451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:05.012417    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:05.411094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:05.411063    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:05.804963 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:05.804911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:05.866065 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:05.866042    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:06.205570 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:06.205542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:06.409128 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:06.409091    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.017913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.017885    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.119413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.119374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.221239 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.221202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.417122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.417040    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.598673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.598641    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:26:07.601623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.601601    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b2e90124-26b9-40ad-9281-d14a5bcc74e4-metrics-tls\") pod \"dns-default-c4bvb\" (UID: \"b2e90124-26b9-40ad-9281-d14a5bcc74e4\") " pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:26:07.623030 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.623003    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.638916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.638898    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:26:07.808424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.808378    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:07.942624 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.942532    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-c4bvb"]
Sep 19 12:26:07.946485 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:07.946440    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2e90124_26b9_40ad_9281_d14a5bcc74e4.slice/crio-03a74b7fbc17e1f166ad69d8a0d3a420ea763148b283694a5303d3027f54addc WatchSource:0}: Error finding container 03a74b7fbc17e1f166ad69d8a0d3a420ea763148b283694a5303d3027f54addc: Status 404 returned error can't find the container with id 03a74b7fbc17e1f166ad69d8a0d3a420ea763148b283694a5303d3027f54addc
Sep 19 12:26:07.994643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:07.994614    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4bvb" event={"ID":"b2e90124-26b9-40ad-9281-d14a5bcc74e4","Type":"ContainerStarted","Data":"03a74b7fbc17e1f166ad69d8a0d3a420ea763148b283694a5303d3027f54addc"}
Sep 19 12:26:08.409454 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:08.409415    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:08.505343 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:08.505311    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:26:08.507988 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:08.507960    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/29a6e79e-a793-4ed7-825c-284e062e97fc-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qnpdg\" (UID: \"29a6e79e-a793-4ed7-825c-284e062e97fc\") " pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:26:08.562311 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:08.562277    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qnpdg"
Sep 19 12:26:08.815243 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:08.815192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:08.861100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:08.860725    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qnpdg"]
Sep 19 12:26:08.863590 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:08.863561    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a6e79e_a793_4ed7_825c_284e062e97fc.slice/crio-390a33c1c1a393f38dfd27dff6c2f1923d925c165377236e8dd5cf408cbb2265 WatchSource:0}: Error finding container 390a33c1c1a393f38dfd27dff6c2f1923d925c165377236e8dd5cf408cbb2265: Status 404 returned error can't find the container with id 390a33c1c1a393f38dfd27dff6c2f1923d925c165377236e8dd5cf408cbb2265
Sep 19 12:26:09.001190 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:09.001149    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnpdg" event={"ID":"29a6e79e-a793-4ed7-825c-284e062e97fc","Type":"ContainerStarted","Data":"e12bc576c5dd325a1571142f0dbd28e67a1b475245caabeb766432b7dbdb3dde"}
Sep 19 12:26:09.001380 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:09.001201    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnpdg" event={"ID":"29a6e79e-a793-4ed7-825c-284e062e97fc","Type":"ContainerStarted","Data":"390a33c1c1a393f38dfd27dff6c2f1923d925c165377236e8dd5cf408cbb2265"}
Sep 19 12:26:09.043700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:09.043665    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:09.402136 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:09.402097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:09.505839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:09.505797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:10.212302 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:10.212276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:10.417511 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:10.417454    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:10.810355 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:10.809983    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:11.010439 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.010395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4bvb" event={"ID":"b2e90124-26b9-40ad-9281-d14a5bcc74e4","Type":"ContainerStarted","Data":"1de725307b692040d04f0f37be1928d6d46211beba87594b2eae3c14e45d261c"}
Sep 19 12:26:11.010616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.010478    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-c4bvb" event={"ID":"b2e90124-26b9-40ad-9281-d14a5bcc74e4","Type":"ContainerStarted","Data":"d640783aefaebaed8436b2d26fb6427c060c841f00aaf09b5fa6ce67513e7903"}
Sep 19 12:26:11.010616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.010524    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:26:11.013069 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.013042    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnpdg" event={"ID":"29a6e79e-a793-4ed7-825c-284e062e97fc","Type":"ContainerStarted","Data":"b43c387989275659da67cd5a75787d90a58eab0d35d41b9dd9110a424ba55771"}
Sep 19 12:26:11.016250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.016182    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:11.032170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.032126    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-c4bvb" podStartSLOduration=18.059764751 podStartE2EDuration="20.032109874s" totalImagesPullingTime="1.972345123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:11.030920151 +0000 UTC m=+54.391163092" watchObservedRunningTime="2026-09-19 12:26:11.032109874 +0000 UTC m=+54.392352815"
Sep 19 12:26:11.215082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:11.215037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:12.008606 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:12.008573    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:12.414635 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:12.414554    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:12.612720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:12.612685    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:12.824094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:12.824059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:13.021231 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:13.021197    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qnpdg" event={"ID":"29a6e79e-a793-4ed7-825c-284e062e97fc","Type":"ContainerStarted","Data":"dc5d6a4c16c810be1057dee777b787c451499fd3c73aeb6c125162ff77804ff9"}
Sep 19 12:26:13.044714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:13.044638    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qnpdg" podStartSLOduration=18.094399344 podStartE2EDuration="21.044578331s" totalImagesPullingTime="2.950178987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 12:25:52 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:13.042693755 +0000 UTC m=+56.402936693" watchObservedRunningTime="2026-09-19 12:26:13.044578331 +0000 UTC m=+56.404821275"
Sep 19 12:26:13.143413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:13.143339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:13.249704 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:13.249672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:15.908334 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:15.908307    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zpcx5"
Sep 19 12:26:16.797983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.797950    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"]
Sep 19 12:26:16.823953 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.823926    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"]
Sep 19 12:26:16.824130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.824054    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.826853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.826822    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 19 12:26:16.827021 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.826999    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 19 12:26:16.827229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.827003    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 19 12:26:16.827328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.827015    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 19 12:26:16.827328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.827074    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 19 12:26:16.827328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.827126    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 19 12:26:16.828147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.828132    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 19 12:26:16.902898 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.902871    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:16.967414 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.967382    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-hub\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.967801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.967425    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.967801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.967500    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnwwr\" (UniqueName: \"kubernetes.io/projected/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-kube-api-access-gnwwr\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.967801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.967539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-ca\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.967801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.967598    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.967801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.967618    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:16.991538 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:16.991515    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:17.068667 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.068579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-ca\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.068667 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.068658    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.068870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.068687    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.068870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.068729    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-hub\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.068870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.068782    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.068870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.068814    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnwwr\" (UniqueName: \"kubernetes.io/projected/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-kube-api-access-gnwwr\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.069646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.069612    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.071316 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.071288    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.071416 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.071314    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-hub\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.071519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.071503    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-ca\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.071800 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.071782    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.078924 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.078903    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnwwr\" (UniqueName: \"kubernetes.io/projected/3d00e463-3746-4d4c-9f18-0c6ac5c2b55c-kube-api-access-gnwwr\") pod \"cluster-proxy-proxy-agent-6bbd5968dc-xk6b7\" (UID: \"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.090687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.090653    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:17.145136 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.145108    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"
Sep 19 12:26:17.189636 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.189598    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:17.295420 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.295371    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:17.439572 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.439548    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7"]
Sep 19 12:26:17.441944 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:17.441908    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d00e463_3746_4d4c_9f18_0c6ac5c2b55c.slice/crio-d302b946afbccfdb387f33472d598cee743a8107317d7e53c9e2d01d4bec1c7c WatchSource:0}: Error finding container d302b946afbccfdb387f33472d598cee743a8107317d7e53c9e2d01d4bec1c7c: Status 404 returned error can't find the container with id d302b946afbccfdb387f33472d598cee743a8107317d7e53c9e2d01d4bec1c7c
Sep 19 12:26:17.993524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:17.993486    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:18.038962 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:18.038931    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7" event={"ID":"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c","Type":"ContainerStarted","Data":"d302b946afbccfdb387f33472d598cee743a8107317d7e53c9e2d01d4bec1c7c"}
Sep 19 12:26:18.092516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:18.092480    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:18.314994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:18.314911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:18.379182 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:18.379144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:19.370832 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:19.370787    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:19.467853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:19.467822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:19.550033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:19.549997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:19.907572 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:19.907529    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:20.071725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:20.071635    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:20.171693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:20.171605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:20.272295 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:20.272252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:20.334699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:20.334655    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:20.397429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:20.397381    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:21.017956 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:21.017930    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-c4bvb"
Sep 19 12:26:22.055691 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:22.055654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7" event={"ID":"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c","Type":"ContainerStarted","Data":"56e9c8e66eaa7420ac1413029a07f7b6ed3944e956e82fe53f3cb8f35520c980"}
Sep 19 12:26:22.056067 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:22.055704    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7" event={"ID":"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c","Type":"ContainerStarted","Data":"274430c348cdf0254f08b52e2419c9fa1d5d932bd4bb0d955e87de623b52b65e"}
Sep 19 12:26:22.056067 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:22.055724    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7" event={"ID":"3d00e463-3746-4d4c-9f18-0c6ac5c2b55c","Type":"ContainerStarted","Data":"bea6938bdae0647ef21f8f3abb8a8aa13745bbf89c3a5f9e3d966fce6319b531"}
Sep 19 12:26:22.077450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:22.077408    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6bbd5968dc-xk6b7" podStartSLOduration=2.0632289520000002 podStartE2EDuration="6.077393757s" totalImagesPullingTime="4.014164805s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:22.07600072 +0000 UTC m=+65.436243661" watchObservedRunningTime="2026-09-19 12:26:22.077393757 +0000 UTC m=+65.437636697"
Sep 19 12:26:23.551151 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.551112    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:23.589697 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.589668    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"]
Sep 19 12:26:23.591868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.591852    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.594662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.594633    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 19 12:26:23.595824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.595785    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-k6wtm\""
Sep 19 12:26:23.595946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.595838    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 19 12:26:23.595946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.595842    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 19 12:26:23.595946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.595842    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 19 12:26:23.595946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.595868    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 19 12:26:23.601019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.600990    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"]
Sep 19 12:26:23.617620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.617596    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79147762-c8d6-4db2-a12e-64739ddfd95b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.617757 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.617631    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79147762-c8d6-4db2-a12e-64739ddfd95b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.617757 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.617680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxvnf\" (UniqueName: \"kubernetes.io/projected/79147762-c8d6-4db2-a12e-64739ddfd95b-kube-api-access-kxvnf\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.617757 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.617741    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/79147762-c8d6-4db2-a12e-64739ddfd95b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.629835 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.629803    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:23.718335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.718303    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxvnf\" (UniqueName: \"kubernetes.io/projected/79147762-c8d6-4db2-a12e-64739ddfd95b-kube-api-access-kxvnf\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.718483 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.718341    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/79147762-c8d6-4db2-a12e-64739ddfd95b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.718483 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.718394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79147762-c8d6-4db2-a12e-64739ddfd95b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.718483 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.718423    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79147762-c8d6-4db2-a12e-64739ddfd95b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.719184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.719160    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/79147762-c8d6-4db2-a12e-64739ddfd95b-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.720987 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.720957    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/79147762-c8d6-4db2-a12e-64739ddfd95b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.721079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.721064    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/79147762-c8d6-4db2-a12e-64739ddfd95b-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.742162 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.742133    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxvnf\" (UniqueName: \"kubernetes.io/projected/79147762-c8d6-4db2-a12e-64739ddfd95b-kube-api-access-kxvnf\") pod \"prometheus-operator-7bd5d56d6-kq4jv\" (UID: \"79147762-c8d6-4db2-a12e-64739ddfd95b\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:23.767662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.767630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:23.900593 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:23.900517    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"
Sep 19 12:26:24.166184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:24.166114    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv"]
Sep 19 12:26:24.168813 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:24.168783    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79147762_c8d6_4db2_a12e_64739ddfd95b.slice/crio-45c35d8cf0d6c1d518cc1bbd95193198b27df24d0ba54c888f3a0800936f2918 WatchSource:0}: Error finding container 45c35d8cf0d6c1d518cc1bbd95193198b27df24d0ba54c888f3a0800936f2918: Status 404 returned error can't find the container with id 45c35d8cf0d6c1d518cc1bbd95193198b27df24d0ba54c888f3a0800936f2918
Sep 19 12:26:24.524873 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:24.524822    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:26:24.527170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:24.527144    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/121c696e-d742-4a44-beb3-2caf250a1b96-metrics-certs\") pod \"network-metrics-daemon-4569q\" (UID: \"121c696e-d742-4a44-beb3-2caf250a1b96\") " pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:26:24.762662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:24.762634    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-g8nvf\""
Sep 19 12:26:24.770962 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:24.770944    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4569q"
Sep 19 12:26:25.036813 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:25.036736    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4569q"]
Sep 19 12:26:25.040496 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:25.040449    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod121c696e_d742_4a44_beb3_2caf250a1b96.slice/crio-e071e8a2dafcc4eb205c689b40e7c578905863f15a4d63cbfd16c3ab0e37ec23 WatchSource:0}: Error finding container e071e8a2dafcc4eb205c689b40e7c578905863f15a4d63cbfd16c3ab0e37ec23: Status 404 returned error can't find the container with id e071e8a2dafcc4eb205c689b40e7c578905863f15a4d63cbfd16c3ab0e37ec23
Sep 19 12:26:25.067333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:25.067303    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv" event={"ID":"79147762-c8d6-4db2-a12e-64739ddfd95b","Type":"ContainerStarted","Data":"45c35d8cf0d6c1d518cc1bbd95193198b27df24d0ba54c888f3a0800936f2918"}
Sep 19 12:26:25.068731 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:25.068687    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4569q" event={"ID":"121c696e-d742-4a44-beb3-2caf250a1b96","Type":"ContainerStarted","Data":"e071e8a2dafcc4eb205c689b40e7c578905863f15a4d63cbfd16c3ab0e37ec23"}
Sep 19 12:26:25.855021 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:25.854938    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:25.996301 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:25.996212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.074376 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.074345    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv" event={"ID":"79147762-c8d6-4db2-a12e-64739ddfd95b","Type":"ContainerStarted","Data":"49d7c1d78790d1cf41c36f5a3a76277c76bf3cf021ba425a203fc1472c8f0eb6"}
Sep 19 12:26:26.074566 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.074403    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv" event={"ID":"79147762-c8d6-4db2-a12e-64739ddfd95b","Type":"ContainerStarted","Data":"646ade060a7584876875cb545824b29d606f48dc8baf6374bad63afae69cfc9e"}
Sep 19 12:26:26.092976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.092948    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.093224 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.093185    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-kq4jv" podStartSLOduration=1.699470492 podStartE2EDuration="3.093170028s" totalImagesPullingTime="1.393699536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:26.092152189 +0000 UTC m=+69.452395134" watchObservedRunningTime="2026-09-19 12:26:26.093170028 +0000 UTC m=+69.453412971"
Sep 19 12:26:26.192088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.192055    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.292692 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.292664    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.357562 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.357521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.467647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.467612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.867645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.867537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:26.930901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:26.930868    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.071838 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.071802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.079180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.079151    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4569q" event={"ID":"121c696e-d742-4a44-beb3-2caf250a1b96","Type":"ContainerStarted","Data":"cab694f98862c8d5a2045b1a14d2d5a268305950c3ddeb83dfd38e1d1492e3b3"}
Sep 19 12:26:27.079295 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.079208    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4569q" event={"ID":"121c696e-d742-4a44-beb3-2caf250a1b96","Type":"ContainerStarted","Data":"42056df5974bc7db0824aae441741ce0764fcb13a60e8631e0b31c329b067502"}
Sep 19 12:26:27.099152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.099107    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4569q" podStartSLOduration=68.057226728 podStartE2EDuration="1m9.099092002s" totalImagesPullingTime="1.041865274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:25:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:27.09735769 +0000 UTC m=+70.457600632" watchObservedRunningTime="2026-09-19 12:26:27.099092002 +0000 UTC m=+70.459334943"
Sep 19 12:26:27.172925 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.172849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.273686 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.273650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.373049 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.373015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.708773 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.708740    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.810441 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.810409    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:27.909926 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:27.909895    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:28.006938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.006858    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cbg5g"]
Sep 19 12:26:28.009174 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.009156    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.012440 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.012417    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 19 12:26:28.013004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.012985    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-f82qw\""
Sep 19 12:26:28.013249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.013236    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 19 12:26:28.013606 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.013582    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 19 12:26:28.022074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.022049    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:28.052523 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052501    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-wtmp\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052529    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-root\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052551    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-textfile\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052589    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-sys\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052615    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-tls\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052634    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052895 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052710    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e243b504-1c74-46e3-9eae-f0901dfe5fce-metrics-client-ca\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052895 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052726    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzdfq\" (UniqueName: \"kubernetes.io/projected/e243b504-1c74-46e3-9eae-f0901dfe5fce-kube-api-access-jzdfq\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.052895 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.052755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.117220 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.117192    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:28.153874 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153853    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-wtmp\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.153977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153881    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-root\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.153977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153901    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-textfile\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.153977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-sys\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.153977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153957    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-tls\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153978    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-root\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154000    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-sys\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.153987    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154031    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-wtmp\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154061    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e243b504-1c74-46e3-9eae-f0901dfe5fce-metrics-client-ca\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154170    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzdfq\" (UniqueName: \"kubernetes.io/projected/e243b504-1c74-46e3-9eae-f0901dfe5fce-kube-api-access-jzdfq\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154277    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-textfile\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154363    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154648 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154625    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e243b504-1c74-46e3-9eae-f0901dfe5fce-metrics-client-ca\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.154718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.154656    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-accelerators-collector-config\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.156289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.156266    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-tls\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.156560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.156538    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e243b504-1c74-46e3-9eae-f0901dfe5fce-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.162913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.162895    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzdfq\" (UniqueName: \"kubernetes.io/projected/e243b504-1c74-46e3-9eae-f0901dfe5fce-kube-api-access-jzdfq\") pod \"node-exporter-cbg5g\" (UID: \"e243b504-1c74-46e3-9eae-f0901dfe5fce\") " pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.216143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.216121    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:28.318369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.318313    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cbg5g"
Sep 19 12:26:28.328022 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:28.328002    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode243b504_1c74_46e3_9eae_f0901dfe5fce.slice/crio-e926e289a54db97b12ca33ab832d60e56dac8a9435122f271169b500ea91f3e2 WatchSource:0}: Error finding container e926e289a54db97b12ca33ab832d60e56dac8a9435122f271169b500ea91f3e2: Status 404 returned error can't find the container with id e926e289a54db97b12ca33ab832d60e56dac8a9435122f271169b500ea91f3e2
Sep 19 12:26:28.426753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.426721    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:28.567232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:28.567194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:29.017704 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.017666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:29.089426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.089393    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbg5g" event={"ID":"e243b504-1c74-46e3-9eae-f0901dfe5fce","Type":"ContainerStarted","Data":"e926e289a54db97b12ca33ab832d60e56dac8a9435122f271169b500ea91f3e2"}
Sep 19 12:26:29.112637 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.112607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:29.230083 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.230044    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:29.530791 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.530752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:29.631189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.631157    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:29.731276 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:29.731250    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:30.096330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:30.096170    2571 generic.go:356] "Generic (PLEG): container finished" podID="e243b504-1c74-46e3-9eae-f0901dfe5fce" containerID="89a28a078b3ce124bcdd68a3e222535be8a99ac8b9de23c93cfa8821f6c91dbf" exitCode=0
Sep 19 12:26:30.096330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:30.096204    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbg5g" event={"ID":"e243b504-1c74-46e3-9eae-f0901dfe5fce","Type":"ContainerDied","Data":"89a28a078b3ce124bcdd68a3e222535be8a99ac8b9de23c93cfa8821f6c91dbf"}
Sep 19 12:26:30.937790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:30.937750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:31.041717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:31.041683    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:31.101446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:31.101419    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbg5g" event={"ID":"e243b504-1c74-46e3-9eae-f0901dfe5fce","Type":"ContainerStarted","Data":"5476f91a2254c7730cc463d622a3419471d80d8c5d0148c0f9107d060bb60c2b"}
Sep 19 12:26:31.101818 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:31.101490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cbg5g" event={"ID":"e243b504-1c74-46e3-9eae-f0901dfe5fce","Type":"ContainerStarted","Data":"a58ad31556827c357595d7b7f8a95100b98f44b137977005c694be39ecf6b1c6"}
Sep 19 12:26:31.121397 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:31.121368    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:31.126897 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:31.126857    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cbg5g" podStartSLOduration=2.974838236 podStartE2EDuration="4.126844004s" totalImagesPullingTime="1.091018535s" totalInitContainerRuntime="60.987233ms" isStatefulPod=true podCreationTimestamp="2026-09-19 12:26:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:31.125653931 +0000 UTC m=+74.485896873" watchObservedRunningTime="2026-09-19 12:26:31.126844004 +0000 UTC m=+74.487086947"
Sep 19 12:26:31.977118 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:31.977091    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-c2nxz"
Sep 19 12:26:32.467570 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.467541    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7894ffc44b-5kv7h"]
Sep 19 12:26:32.493495 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.493454    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7894ffc44b-5kv7h"]
Sep 19 12:26:32.493619 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.493544    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.496782 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.496756    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 19 12:26:32.496898 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.496823    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-ts6p7\""
Sep 19 12:26:32.498280 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.498263    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 19 12:26:32.498399 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.498382    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 19 12:26:32.498478 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.498418    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 19 12:26:32.498524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.498475    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-amcadtd4ek0n7\""
Sep 19 12:26:32.588465 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588442    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-secret-metrics-server-tls\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.588539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588484    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-metrics-server-audit-profiles\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.588539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588507    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-secret-metrics-server-client-certs\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.588614 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588584    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-kube-api-access-dh9gj\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.588614 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-audit-log\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.588678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588632    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.588717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.588700    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-client-ca-bundle\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.689937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.689906    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-audit-log\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690066 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.689962    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690066 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690012    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-client-ca-bundle\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690066 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690039    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-secret-metrics-server-tls\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690065    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-metrics-server-audit-profiles\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690093    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-secret-metrics-server-client-certs\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690145    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-kube-api-access-dh9gj\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690367    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-audit-log\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.690857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.690807    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.691364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.691318    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-metrics-server-audit-profiles\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.692849 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.692824    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-client-ca-bundle\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.692959 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.692940    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-secret-metrics-server-tls\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.693118 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.693101    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-secret-metrics-server-client-certs\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.702812 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.702787    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh9gj\" (UniqueName: \"kubernetes.io/projected/b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b-kube-api-access-dh9gj\") pod \"metrics-server-7894ffc44b-5kv7h\" (UID: \"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b\") " pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.719607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.719553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:32.802517 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.802495    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:32.816350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.816322    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:32.916196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:32.916167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:33.098579 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:33.098533    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7894ffc44b-5kv7h"]
Sep 19 12:26:33.101409 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:33.101381    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1f1ddff_9e56_4ae8_9ef2_8dcb2a86ce2b.slice/crio-267ae9791edb0fff7478c0b150747a4f410ec6726f5ce4715aad2ed7f366a7ac WatchSource:0}: Error finding container 267ae9791edb0fff7478c0b150747a4f410ec6726f5ce4715aad2ed7f366a7ac: Status 404 returned error can't find the container with id 267ae9791edb0fff7478c0b150747a4f410ec6726f5ce4715aad2ed7f366a7ac
Sep 19 12:26:33.109553 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:33.109530    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h" event={"ID":"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b","Type":"ContainerStarted","Data":"267ae9791edb0fff7478c0b150747a4f410ec6726f5ce4715aad2ed7f366a7ac"}
Sep 19 12:26:34.163040 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:34.163005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:34.242147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:34.242095    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:35.118088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:35.118050    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h" event={"ID":"b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b","Type":"ContainerStarted","Data":"760b4aad3f161243e18136856ba647ac8ebd8b42aca72d0ccac0282ceebbd4f6"}
Sep 19 12:26:35.137069 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:35.137019    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h" podStartSLOduration=1.651670491 podStartE2EDuration="3.13700314s" totalImagesPullingTime="1.485332649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:35.136401 +0000 UTC m=+78.496643942" watchObservedRunningTime="2026-09-19 12:26:35.13700314 +0000 UTC m=+78.497246080"
Sep 19 12:26:36.561056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.561024    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d64868587-tlqc5"]
Sep 19 12:26:36.564124 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.564108    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.566724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.566699    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4pd8c\""
Sep 19 12:26:36.566850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.566755    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 12:26:36.566850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.566710    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 12:26:36.567952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.567928    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 12:26:36.568066 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.568009    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 12:26:36.568066 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.568020    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 12:26:36.568188 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.568085    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 12:26:36.568188 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.568166    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 12:26:36.575296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.575272    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d64868587-tlqc5"]
Sep 19 12:26:36.618265 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.618235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-oauth-serving-cert\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.618453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.618291    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-serving-cert\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.618453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.618358    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-service-ca\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.618453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.618402    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dphfq\" (UniqueName: \"kubernetes.io/projected/59443281-2373-4c59-b7f6-2d2281e468bc-kube-api-access-dphfq\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.618453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.618429    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-console-config\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.618629 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.618516    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-oauth-config\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.701708 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.701670    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:36.719848 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.719823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-serving-cert\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.719926 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.719866    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-service-ca\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.719926 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.719888    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dphfq\" (UniqueName: \"kubernetes.io/projected/59443281-2373-4c59-b7f6-2d2281e468bc-kube-api-access-dphfq\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.719926 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.719910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-console-config\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.720054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.719936    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-oauth-config\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.720054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.719955    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-oauth-serving-cert\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.720648 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.720621    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-service-ca\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.720740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.720632    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-console-config\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.720740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.720700    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-oauth-serving-cert\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.722948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.722922    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-serving-cert\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.723047 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.723027    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-oauth-config\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.731033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.731004    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dphfq\" (UniqueName: \"kubernetes.io/projected/59443281-2373-4c59-b7f6-2d2281e468bc-kube-api-access-dphfq\") pod \"console-5d64868587-tlqc5\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") " pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:36.771376 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.771347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:36.873093 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:36.873005    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:37.148051 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:37.147976    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d64868587-tlqc5"]
Sep 19 12:26:37.150535 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:26:37.150510    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59443281_2373_4c59_b7f6_2d2281e468bc.slice/crio-13595feb18e1b15b3f2d238b55b111af1a7620c00b2d465fdda0091ae16a5413 WatchSource:0}: Error finding container 13595feb18e1b15b3f2d238b55b111af1a7620c00b2d465fdda0091ae16a5413: Status 404 returned error can't find the container with id 13595feb18e1b15b3f2d238b55b111af1a7620c00b2d465fdda0091ae16a5413
Sep 19 12:26:38.089845 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:38.089813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:38.133386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:38.133347    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d64868587-tlqc5" event={"ID":"59443281-2373-4c59-b7f6-2d2281e468bc","Type":"ContainerStarted","Data":"13595feb18e1b15b3f2d238b55b111af1a7620c00b2d465fdda0091ae16a5413"}
Sep 19 12:26:38.152423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:38.152366    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:39.121950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:39.121884    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:39.222285 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:39.222238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:39.422736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:39.422645    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:42.151313 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:42.151274    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d64868587-tlqc5" event={"ID":"59443281-2373-4c59-b7f6-2d2281e468bc","Type":"ContainerStarted","Data":"5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2"}
Sep 19 12:26:42.169982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:42.169934    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d64868587-tlqc5" podStartSLOduration=2.076339957 podStartE2EDuration="6.169919988s" totalImagesPullingTime="4.093580031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:26:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:26:42.168855045 +0000 UTC m=+85.529097986" watchObservedRunningTime="2026-09-19 12:26:42.169919988 +0000 UTC m=+85.530162929"
Sep 19 12:26:42.379541 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:42.379499    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:42.477739 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:42.477706    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:42.900748 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:42.900656    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:42.963694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:42.963660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:43.130680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:43.130645    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:43.232194 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:43.232163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:46.873836 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:46.873803    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:46.874300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:46.873867    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:46.878427 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:46.878406    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:47.154115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:47.154029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:47.172643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:47.172619    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:26:47.256580 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:47.256539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:47.709539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:47.709502    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:47.810860 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:47.810826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:51.507245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:51.507210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:51.611567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:51.611536    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:52.802792 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:52.802753    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:52.803188 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:52.802820    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:26:54.304680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:54.304643    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:54.361580 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:54.361547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:57.297694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:57.297662    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d64868587-tlqc5"]
Sep 19 12:26:58.130606 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:58.130570    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:26:58.190164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:26:58.190131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:01.104968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:01.104937    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:01.161183 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:01.161143    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:05.274402 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:05.274359    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:05.374921 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:05.374883    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:05.437375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:05.437337    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:12.808737 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:12.808707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:27:12.812735 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:12.812707    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7894ffc44b-5kv7h"
Sep 19 12:27:13.137004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:13.136921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:13.237422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:13.237392    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:22.316168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.316110    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d64868587-tlqc5" podUID="59443281-2373-4c59-b7f6-2d2281e468bc" containerName="console" containerID="cri-o://5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2" gracePeriod=15
Sep 19 12:27:22.544899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.544876    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:27:22.661360 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661265    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-oauth-config\") pod \"59443281-2373-4c59-b7f6-2d2281e468bc\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") "
Sep 19 12:27:22.661360 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661305    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-console-config\") pod \"59443281-2373-4c59-b7f6-2d2281e468bc\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") "
Sep 19 12:27:22.661360 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661335    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-serving-cert\") pod \"59443281-2373-4c59-b7f6-2d2281e468bc\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") "
Sep 19 12:27:22.661661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661382    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dphfq\" (UniqueName: \"kubernetes.io/projected/59443281-2373-4c59-b7f6-2d2281e468bc-kube-api-access-dphfq\") pod \"59443281-2373-4c59-b7f6-2d2281e468bc\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") "
Sep 19 12:27:22.661661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661420    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-service-ca\") pod \"59443281-2373-4c59-b7f6-2d2281e468bc\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") "
Sep 19 12:27:22.661661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661443    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-oauth-serving-cert\") pod \"59443281-2373-4c59-b7f6-2d2281e468bc\" (UID: \"59443281-2373-4c59-b7f6-2d2281e468bc\") "
Sep 19 12:27:22.661869 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661839    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-console-config" (OuterVolumeSpecName: "console-config") pod "59443281-2373-4c59-b7f6-2d2281e468bc" (UID: "59443281-2373-4c59-b7f6-2d2281e468bc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:22.661982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661868    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "59443281-2373-4c59-b7f6-2d2281e468bc" (UID: "59443281-2373-4c59-b7f6-2d2281e468bc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:22.661982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.661835    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-service-ca" (OuterVolumeSpecName: "service-ca") pod "59443281-2373-4c59-b7f6-2d2281e468bc" (UID: "59443281-2373-4c59-b7f6-2d2281e468bc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:27:22.663663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.663635    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "59443281-2373-4c59-b7f6-2d2281e468bc" (UID: "59443281-2373-4c59-b7f6-2d2281e468bc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:22.663663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.663647    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59443281-2373-4c59-b7f6-2d2281e468bc-kube-api-access-dphfq" (OuterVolumeSpecName: "kube-api-access-dphfq") pod "59443281-2373-4c59-b7f6-2d2281e468bc" (UID: "59443281-2373-4c59-b7f6-2d2281e468bc"). InnerVolumeSpecName "kube-api-access-dphfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:27:22.663811 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.663649    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "59443281-2373-4c59-b7f6-2d2281e468bc" (UID: "59443281-2373-4c59-b7f6-2d2281e468bc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:27:22.762642 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.762613    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-serving-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:27:22.762642 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.762642    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dphfq\" (UniqueName: \"kubernetes.io/projected/59443281-2373-4c59-b7f6-2d2281e468bc-kube-api-access-dphfq\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:27:22.762830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.762654    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:27:22.762830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.762662    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-oauth-serving-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:27:22.762830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.762671    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/59443281-2373-4c59-b7f6-2d2281e468bc-console-oauth-config\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:27:22.762830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:22.762678    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/59443281-2373-4c59-b7f6-2d2281e468bc-console-config\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:27:23.291057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.291021    2571 generic.go:356] "Generic (PLEG): container finished" podID="59443281-2373-4c59-b7f6-2d2281e468bc" containerID="5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2" exitCode=0
Sep 19 12:27:23.291218 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.291078    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d64868587-tlqc5"
Sep 19 12:27:23.291218 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.291110    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d64868587-tlqc5" event={"ID":"59443281-2373-4c59-b7f6-2d2281e468bc","Type":"ContainerDied","Data":"5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2"}
Sep 19 12:27:23.291218 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.291193    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d64868587-tlqc5" event={"ID":"59443281-2373-4c59-b7f6-2d2281e468bc","Type":"ContainerDied","Data":"13595feb18e1b15b3f2d238b55b111af1a7620c00b2d465fdda0091ae16a5413"}
Sep 19 12:27:23.291327 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.291230    2571 scope.go:117] "RemoveContainer" containerID="5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2"
Sep 19 12:27:23.300293 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.300274    2571 scope.go:117] "RemoveContainer" containerID="5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2"
Sep 19 12:27:23.300584 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:27:23.300558    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2\": container with ID starting with 5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2 not found: ID does not exist" containerID="5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2"
Sep 19 12:27:23.300644 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.300596    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2"} err="failed to get container status \"5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2\": rpc error: code = NotFound desc = could not find container \"5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2\": container with ID starting with 5f956ab67d69a6014626a7ba664b7e2559f4ca94adeccec20b0740fefed15cb2 not found: ID does not exist"
Sep 19 12:27:23.308662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.308641    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d64868587-tlqc5"]
Sep 19 12:27:23.312593 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:23.312571    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d64868587-tlqc5"]
Sep 19 12:27:24.741049 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:24.741013    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59443281-2373-4c59-b7f6-2d2281e468bc" path="/var/lib/kubelet/pods/59443281-2373-4c59-b7f6-2d2281e468bc/volumes"
Sep 19 12:27:47.195650 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:47.195617    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:47.396645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:47.396605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:55.704159 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.704121    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f556db87c-g4jzj"]
Sep 19 12:27:55.704678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.704624    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59443281-2373-4c59-b7f6-2d2281e468bc" containerName="console"
Sep 19 12:27:55.704678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.704643    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="59443281-2373-4c59-b7f6-2d2281e468bc" containerName="console"
Sep 19 12:27:55.705730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.704778    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="59443281-2373-4c59-b7f6-2d2281e468bc" containerName="console"
Sep 19 12:27:55.709915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.709893    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.712655 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.712637    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 19 12:27:55.713890 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.713875    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 19 12:27:55.714204 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.714185    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 19 12:27:55.714204 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.714198    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 19 12:27:55.714430 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.714417    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 19 12:27:55.714547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.714529    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4pd8c\""
Sep 19 12:27:55.714640 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.714529    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 19 12:27:55.714694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.714641    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 19 12:27:55.718800 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.718780    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 19 12:27:55.722859 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.722838    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f556db87c-g4jzj"]
Sep 19 12:27:55.769366 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.769336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:55.792227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792205    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-service-ca\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.792309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792229    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-trusted-ca-bundle\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.792309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792255    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq4h4\" (UniqueName: \"kubernetes.io/projected/54f9f527-e828-480c-bfd5-3177244b4fd2-kube-api-access-kq4h4\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.792309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-console-config\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.792404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792330    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-oauth-config\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.792404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792361    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-oauth-serving-cert\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.792404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.792395    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-serving-cert\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.826141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.826117    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:27:55.893181 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893159    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-service-ca\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.893251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893187    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-trusted-ca-bundle\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.893251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893215    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq4h4\" (UniqueName: \"kubernetes.io/projected/54f9f527-e828-480c-bfd5-3177244b4fd2-kube-api-access-kq4h4\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.893315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893254    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-console-config\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.893315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893282    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-oauth-config\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.893315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893310    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-oauth-serving-cert\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.893451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.893355    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-serving-cert\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.894054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.894030    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-console-config\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.894139 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.894095    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-oauth-serving-cert\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.894139 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.894101    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-service-ca\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.894257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.894209    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-trusted-ca-bundle\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.895713 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.895683    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-oauth-config\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.895787 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.895777    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-serving-cert\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:55.901970 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:55.901945    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq4h4\" (UniqueName: \"kubernetes.io/projected/54f9f527-e828-480c-bfd5-3177244b4fd2-kube-api-access-kq4h4\") pod \"console-f556db87c-g4jzj\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") " pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:56.018133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:56.018082    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:27:56.289871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:56.289843    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f556db87c-g4jzj"]
Sep 19 12:27:56.292159 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:27:56.292132    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54f9f527_e828_480c_bfd5_3177244b4fd2.slice/crio-b3ac6889a00a55f6e8d69321e2d8393cd4d74587959df50c1e054217376916aa WatchSource:0}: Error finding container b3ac6889a00a55f6e8d69321e2d8393cd4d74587959df50c1e054217376916aa: Status 404 returned error can't find the container with id b3ac6889a00a55f6e8d69321e2d8393cd4d74587959df50c1e054217376916aa
Sep 19 12:27:56.404124 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:56.404077    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f556db87c-g4jzj" event={"ID":"54f9f527-e828-480c-bfd5-3177244b4fd2","Type":"ContainerStarted","Data":"fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d"}
Sep 19 12:27:56.404297 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:56.404134    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f556db87c-g4jzj" event={"ID":"54f9f527-e828-480c-bfd5-3177244b4fd2","Type":"ContainerStarted","Data":"b3ac6889a00a55f6e8d69321e2d8393cd4d74587959df50c1e054217376916aa"}
Sep 19 12:27:56.423686 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:27:56.423640    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f556db87c-g4jzj" podStartSLOduration=1.4236233409999999 podStartE2EDuration="1.423623341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:27:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:27:56.422941667 +0000 UTC m=+159.783184609" watchObservedRunningTime="2026-09-19 12:27:56.423623341 +0000 UTC m=+159.783866282"
Sep 19 12:28:06.018742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.018653    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:28:06.018742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.018740    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:28:06.023162 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.023142    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:28:06.441817 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.441740    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:28:06.607393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.607362    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:28:06.709432 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.709397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:28:06.777927 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:06.777898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:28:35.992232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:35.992190    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:28:36.763054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:36.763010    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:28:48.380406 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:48.380374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:28:48.476977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:28:48.476938    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:02.433508 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:02.433449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:02.486174 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:02.486130    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:13.586189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:13.586120    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:14.325578 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:14.325535    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:14.378095 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:14.378061    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:23.933075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:23.933040    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f556db87c-g4jzj"]
Sep 19 12:29:24.029387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:24.029356    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:24.131937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:24.131907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:24.186743 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:24.186677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:28.149938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:28.149896    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:28.239000 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:28.238968    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:28.432399 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:28.432327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:38.704944 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:38.704860    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:48.937805 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:48.937757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:48.950914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:48.950877    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f556db87c-g4jzj" podUID="54f9f527-e828-480c-bfd5-3177244b4fd2" containerName="console" containerID="cri-o://fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d" gracePeriod=15
Sep 19 12:29:49.189895 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.189837    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:29:49.253432 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.253398    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:49.301169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.301136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:49.302042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302023    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kq4h4\" (UniqueName: \"kubernetes.io/projected/54f9f527-e828-480c-bfd5-3177244b4fd2-kube-api-access-kq4h4\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302099 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302065    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-trusted-ca-bundle\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302099 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302089    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-oauth-config\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302276 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302259    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-serving-cert\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302291    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-console-config\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302319    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-oauth-serving-cert\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302490 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302449    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:29:49.302490 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302479    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-service-ca\") pod \"54f9f527-e828-480c-bfd5-3177244b4fd2\" (UID: \"54f9f527-e828-480c-bfd5-3177244b4fd2\") "
Sep 19 12:29:49.302725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302693    2571 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-trusted-ca-bundle\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.302725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302699    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:29:49.302725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302706    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-console-config" (OuterVolumeSpecName: "console-config") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:29:49.302884 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.302864    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-service-ca" (OuterVolumeSpecName: "service-ca") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:29:49.304361 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.304340    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:29:49.304729 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.304708    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54f9f527-e828-480c-bfd5-3177244b4fd2-kube-api-access-kq4h4" (OuterVolumeSpecName: "kube-api-access-kq4h4") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "kube-api-access-kq4h4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:29:49.304810 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.304785    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "54f9f527-e828-480c-bfd5-3177244b4fd2" (UID: "54f9f527-e828-480c-bfd5-3177244b4fd2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:29:49.403307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.403262    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kq4h4\" (UniqueName: \"kubernetes.io/projected/54f9f527-e828-480c-bfd5-3177244b4fd2-kube-api-access-kq4h4\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.403307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.403301    2571 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-oauth-config\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.403307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.403310    2571 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/54f9f527-e828-480c-bfd5-3177244b4fd2-console-serving-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.403307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.403319    2571 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-console-config\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.403587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.403328    2571 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-oauth-serving-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.403587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.403336    2571 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/54f9f527-e828-480c-bfd5-3177244b4fd2-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:29:49.779411 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.779376    2571 generic.go:356] "Generic (PLEG): container finished" podID="54f9f527-e828-480c-bfd5-3177244b4fd2" containerID="fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d" exitCode=0
Sep 19 12:29:49.779608 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.779421    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f556db87c-g4jzj"
Sep 19 12:29:49.779608 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.779445    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f556db87c-g4jzj" event={"ID":"54f9f527-e828-480c-bfd5-3177244b4fd2","Type":"ContainerDied","Data":"fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d"}
Sep 19 12:29:49.779608 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.779509    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f556db87c-g4jzj" event={"ID":"54f9f527-e828-480c-bfd5-3177244b4fd2","Type":"ContainerDied","Data":"b3ac6889a00a55f6e8d69321e2d8393cd4d74587959df50c1e054217376916aa"}
Sep 19 12:29:49.779608 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.779536    2571 scope.go:117] "RemoveContainer" containerID="fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d"
Sep 19 12:29:49.787444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.787428    2571 scope.go:117] "RemoveContainer" containerID="fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d"
Sep 19 12:29:49.787734 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:29:49.787716    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d\": container with ID starting with fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d not found: ID does not exist" containerID="fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d"
Sep 19 12:29:49.787795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.787747    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d"} err="failed to get container status \"fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d\": rpc error: code = NotFound desc = could not find container \"fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d\": container with ID starting with fe1330dc7d9aea4a263f599f22869fe83f9c46750354d7a61c4b7d8f39bcc76d not found: ID does not exist"
Sep 19 12:29:49.799096 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.799072    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f556db87c-g4jzj"]
Sep 19 12:29:49.801199 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:49.801180    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f556db87c-g4jzj"]
Sep 19 12:29:50.738315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:50.738280    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54f9f527-e828-480c-bfd5-3177244b4fd2" path="/var/lib/kubelet/pods/54f9f527-e828-480c-bfd5-3177244b4fd2/volumes"
Sep 19 12:29:52.677056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:52.677018    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:52.766710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:52.766675    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:29:53.284418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.284384    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"]
Sep 19 12:29:53.284742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.284723    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54f9f527-e828-480c-bfd5-3177244b4fd2" containerName="console"
Sep 19 12:29:53.284821 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.284743    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="54f9f527-e828-480c-bfd5-3177244b4fd2" containerName="console"
Sep 19 12:29:53.284821 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.284810    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="54f9f527-e828-480c-bfd5-3177244b4fd2" containerName="console"
Sep 19 12:29:53.289004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.288984    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.291821 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.291804    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 12:29:53.291957 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.291941    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 12:29:53.293051 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.293031    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2v6sm\""
Sep 19 12:29:53.298979 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.298961    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"]
Sep 19 12:29:53.431278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.431258    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.431390 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.431293    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dxgl8\" (UniqueName: \"kubernetes.io/projected/9fd06892-513a-4d9a-9f76-35fda2707108-kube-api-access-dxgl8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.431390 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.431322    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.532421 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.532396    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.532535 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.532432    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dxgl8\" (UniqueName: \"kubernetes.io/projected/9fd06892-513a-4d9a-9f76-35fda2707108-kube-api-access-dxgl8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.532535 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.532480    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.532774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.532756    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.532853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.532838    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.541283 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.541232    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dxgl8\" (UniqueName: \"kubernetes.io/projected/9fd06892-513a-4d9a-9f76-35fda2707108-kube-api-access-dxgl8\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.597429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.597406    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:29:53.865215 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:53.865131    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"]
Sep 19 12:29:53.867983 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:29:53.867959    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd06892_513a_4d9a_9f76_35fda2707108.slice/crio-cd618375f2cccb69dd2a76ed4fd40cedffab25fe47844e937be3bc414cffff77 WatchSource:0}: Error finding container cd618375f2cccb69dd2a76ed4fd40cedffab25fe47844e937be3bc414cffff77: Status 404 returned error can't find the container with id cd618375f2cccb69dd2a76ed4fd40cedffab25fe47844e937be3bc414cffff77
Sep 19 12:29:54.796423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:54.796387    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb" event={"ID":"9fd06892-513a-4d9a-9f76-35fda2707108","Type":"ContainerStarted","Data":"cd618375f2cccb69dd2a76ed4fd40cedffab25fe47844e937be3bc414cffff77"}
Sep 19 12:29:59.815765 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:59.815705    2571 generic.go:356] "Generic (PLEG): container finished" podID="9fd06892-513a-4d9a-9f76-35fda2707108" containerID="69a0d7f996bb6b7a59c09679cb91563357dc8e54698031c2777049fa5f7f1ea1" exitCode=0
Sep 19 12:29:59.816151 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:29:59.815789    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb" event={"ID":"9fd06892-513a-4d9a-9f76-35fda2707108","Type":"ContainerDied","Data":"69a0d7f996bb6b7a59c09679cb91563357dc8e54698031c2777049fa5f7f1ea1"}
Sep 19 12:30:01.825724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:01.825692    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb" event={"ID":"9fd06892-513a-4d9a-9f76-35fda2707108","Type":"ContainerStarted","Data":"09c4e966e02a1015c5435770effadab98f410e07a7714d5cb065c9b22b0d3f68"}
Sep 19 12:30:02.830631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:02.830595    2571 generic.go:356] "Generic (PLEG): container finished" podID="9fd06892-513a-4d9a-9f76-35fda2707108" containerID="09c4e966e02a1015c5435770effadab98f410e07a7714d5cb065c9b22b0d3f68" exitCode=0
Sep 19 12:30:02.831106 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:02.830642    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb" event={"ID":"9fd06892-513a-4d9a-9f76-35fda2707108","Type":"ContainerDied","Data":"09c4e966e02a1015c5435770effadab98f410e07a7714d5cb065c9b22b0d3f68"}
Sep 19 12:30:09.858029 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:09.857992    2571 generic.go:356] "Generic (PLEG): container finished" podID="9fd06892-513a-4d9a-9f76-35fda2707108" containerID="cbe289626e5e93b0a75e21a6adc11cfe7ada6a3ea46b1dea3883b8f36891314a" exitCode=0
Sep 19 12:30:09.858390 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:09.858036    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb" event={"ID":"9fd06892-513a-4d9a-9f76-35fda2707108","Type":"ContainerDied","Data":"cbe289626e5e93b0a75e21a6adc11cfe7ada6a3ea46b1dea3883b8f36891314a"}
Sep 19 12:30:10.988792 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:10.988767    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:30:11.068576 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.068543    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-bundle\") pod \"9fd06892-513a-4d9a-9f76-35fda2707108\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") "
Sep 19 12:30:11.068740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.068592    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dxgl8\" (UniqueName: \"kubernetes.io/projected/9fd06892-513a-4d9a-9f76-35fda2707108-kube-api-access-dxgl8\") pod \"9fd06892-513a-4d9a-9f76-35fda2707108\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") "
Sep 19 12:30:11.068740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.068622    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-util\") pod \"9fd06892-513a-4d9a-9f76-35fda2707108\" (UID: \"9fd06892-513a-4d9a-9f76-35fda2707108\") "
Sep 19 12:30:11.069264 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.069236    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-bundle" (OuterVolumeSpecName: "bundle") pod "9fd06892-513a-4d9a-9f76-35fda2707108" (UID: "9fd06892-513a-4d9a-9f76-35fda2707108"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:30:11.070793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.070766    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fd06892-513a-4d9a-9f76-35fda2707108-kube-api-access-dxgl8" (OuterVolumeSpecName: "kube-api-access-dxgl8") pod "9fd06892-513a-4d9a-9f76-35fda2707108" (UID: "9fd06892-513a-4d9a-9f76-35fda2707108"). InnerVolumeSpecName "kube-api-access-dxgl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:30:11.072999 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.072977    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-util" (OuterVolumeSpecName: "util") pod "9fd06892-513a-4d9a-9f76-35fda2707108" (UID: "9fd06892-513a-4d9a-9f76-35fda2707108"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:30:11.169560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.169479    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-bundle\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:30:11.169560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.169507    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dxgl8\" (UniqueName: \"kubernetes.io/projected/9fd06892-513a-4d9a-9f76-35fda2707108-kube-api-access-dxgl8\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:30:11.169560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.169517    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9fd06892-513a-4d9a-9f76-35fda2707108-util\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:30:11.865877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.865842    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb" event={"ID":"9fd06892-513a-4d9a-9f76-35fda2707108","Type":"ContainerDied","Data":"cd618375f2cccb69dd2a76ed4fd40cedffab25fe47844e937be3bc414cffff77"}
Sep 19 12:30:11.865877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.865870    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mz4lb"
Sep 19 12:30:11.866089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:11.865888    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd618375f2cccb69dd2a76ed4fd40cedffab25fe47844e937be3bc414cffff77"
Sep 19 12:30:14.275834 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:14.275801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:15.541666 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:15.541631    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:17.896642 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:17.896604    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:17.991133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:17.991099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:18.645441 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:18.645415    2571 kubelet.go:1648] "Image garbage collection succeeded"
Sep 19 12:30:19.685514 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.685452    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:19.785875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.785844    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:19.864320 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864285    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jktkc"]
Sep 19 12:30:19.864636 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864619    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="extract"
Sep 19 12:30:19.864718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864638    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="extract"
Sep 19 12:30:19.864718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864667    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="pull"
Sep 19 12:30:19.864718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864677    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="pull"
Sep 19 12:30:19.864718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864690    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="util"
Sep 19 12:30:19.864718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864699    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="util"
Sep 19 12:30:19.864949 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.864760    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fd06892-513a-4d9a-9f76-35fda2707108" containerName="extract"
Sep 19 12:30:19.878809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.878783    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jktkc"]
Sep 19 12:30:19.878937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.878878    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:19.883816 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.883793    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 19 12:30:19.884173 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.884158    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-852ql\""
Sep 19 12:30:19.884406 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:19.884390    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 19 12:30:20.038818 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.038785    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15fc6b3-04f3-4352-ae5e-e3a0de046092-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jktkc\" (UID: \"d15fc6b3-04f3-4352-ae5e-e3a0de046092\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.038997 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.038826    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx4fd\" (UniqueName: \"kubernetes.io/projected/d15fc6b3-04f3-4352-ae5e-e3a0de046092-kube-api-access-kx4fd\") pod \"cert-manager-webhook-6486ffd997-jktkc\" (UID: \"d15fc6b3-04f3-4352-ae5e-e3a0de046092\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.139857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.139820    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15fc6b3-04f3-4352-ae5e-e3a0de046092-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jktkc\" (UID: \"d15fc6b3-04f3-4352-ae5e-e3a0de046092\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.139857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.139861    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx4fd\" (UniqueName: \"kubernetes.io/projected/d15fc6b3-04f3-4352-ae5e-e3a0de046092-kube-api-access-kx4fd\") pod \"cert-manager-webhook-6486ffd997-jktkc\" (UID: \"d15fc6b3-04f3-4352-ae5e-e3a0de046092\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.149785 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.149757    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d15fc6b3-04f3-4352-ae5e-e3a0de046092-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-jktkc\" (UID: \"d15fc6b3-04f3-4352-ae5e-e3a0de046092\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.149942 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.149925    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx4fd\" (UniqueName: \"kubernetes.io/projected/d15fc6b3-04f3-4352-ae5e-e3a0de046092-kube-api-access-kx4fd\") pod \"cert-manager-webhook-6486ffd997-jktkc\" (UID: \"d15fc6b3-04f3-4352-ae5e-e3a0de046092\") " pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.188678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.188659    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:20.500081 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.500047    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-jktkc"]
Sep 19 12:30:20.503808 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:30:20.503782    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd15fc6b3_04f3_4352_ae5e_e3a0de046092.slice/crio-7e3f62264f3e8092f02572d5fde6cfc22174e5ffdf418e0544811012bc18f129 WatchSource:0}: Error finding container 7e3f62264f3e8092f02572d5fde6cfc22174e5ffdf418e0544811012bc18f129: Status 404 returned error can't find the container with id 7e3f62264f3e8092f02572d5fde6cfc22174e5ffdf418e0544811012bc18f129
Sep 19 12:30:20.505992 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.505971    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:30:20.538894 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.538857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:20.659818 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.659778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:20.895954 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:20.895874    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc" event={"ID":"d15fc6b3-04f3-4352-ae5e-e3a0de046092","Type":"ContainerStarted","Data":"7e3f62264f3e8092f02572d5fde6cfc22174e5ffdf418e0544811012bc18f129"}
Sep 19 12:30:21.713491 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:21.713445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:22.090714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:22.090637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:24.878695 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:24.878658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:24.934708 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:24.934663    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:26.749130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:26.749081    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:26.849742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:26.849637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:26.919079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:26.919049    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc" event={"ID":"d15fc6b3-04f3-4352-ae5e-e3a0de046092","Type":"ContainerStarted","Data":"2f003e6fad9e604228b12d1860473e50c60d424fdce2c9c84e86deba8a8550c5"}
Sep 19 12:30:26.919189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:26.919160    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:26.936239 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:26.936193    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc" podStartSLOduration=1.924712225 podStartE2EDuration="7.936180373s" totalImagesPullingTime="6.011468148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:30:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:30:26.935225976 +0000 UTC m=+310.295468938" watchObservedRunningTime="2026-09-19 12:30:26.936180373 +0000 UTC m=+310.296423314"
Sep 19 12:30:30.897278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:30.897223    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:32.924576 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:32.924549    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-jktkc"
Sep 19 12:30:42.614578 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:42.614540    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:42.713601 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:42.713569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:48.898560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:48.898525    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:49.002483 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:49.002429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:50.451986 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.451951    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"]
Sep 19 12:30:50.454110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.454095    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.456646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.456622    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 12:30:50.457797 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.457778    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2v6sm\""
Sep 19 12:30:50.457915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.457778    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 12:30:50.462415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.462392    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"]
Sep 19 12:30:50.531852 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.531813    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.532016 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.531864    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2d8wb\" (UniqueName: \"kubernetes.io/projected/6bdde153-67bf-4195-a214-3e10b2d34889-kube-api-access-2d8wb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.532016 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.531917    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.632884 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.632854    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2d8wb\" (UniqueName: \"kubernetes.io/projected/6bdde153-67bf-4195-a214-3e10b2d34889-kube-api-access-2d8wb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.632884 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.632889    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.633035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.632931    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.633323 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.633303    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.633370 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.633316    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.641053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.641019    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2d8wb\" (UniqueName: \"kubernetes.io/projected/6bdde153-67bf-4195-a214-3e10b2d34889-kube-api-access-2d8wb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:50.763513 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:50.763429    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:51.028437 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:51.028367    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"]
Sep 19 12:30:51.031521 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:30:51.031489    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bdde153_67bf_4195_a214_3e10b2d34889.slice/crio-aa91b6114821a589d743959dceb514081dbe1b05a831f16a2f4d71804c9da2c6 WatchSource:0}: Error finding container aa91b6114821a589d743959dceb514081dbe1b05a831f16a2f4d71804c9da2c6: Status 404 returned error can't find the container with id aa91b6114821a589d743959dceb514081dbe1b05a831f16a2f4d71804c9da2c6
Sep 19 12:30:51.032872 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:51.032847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:52.005328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:52.005294    2571 generic.go:356] "Generic (PLEG): container finished" podID="6bdde153-67bf-4195-a214-3e10b2d34889" containerID="2f5dabcca83b63b2212a715e2e9c788456deeac6308082fe49b84bf4a8a8c649" exitCode=0
Sep 19 12:30:52.005690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:52.005366    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz" event={"ID":"6bdde153-67bf-4195-a214-3e10b2d34889","Type":"ContainerDied","Data":"2f5dabcca83b63b2212a715e2e9c788456deeac6308082fe49b84bf4a8a8c649"}
Sep 19 12:30:52.005690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:52.005446    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz" event={"ID":"6bdde153-67bf-4195-a214-3e10b2d34889","Type":"ContainerStarted","Data":"aa91b6114821a589d743959dceb514081dbe1b05a831f16a2f4d71804c9da2c6"}
Sep 19 12:30:53.009867 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:53.009834    2571 generic.go:356] "Generic (PLEG): container finished" podID="6bdde153-67bf-4195-a214-3e10b2d34889" containerID="5ac07b91ed5895246a846260da15d735e59edd09641b605bf5d30f03d8ff92b3" exitCode=0
Sep 19 12:30:53.010254 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:53.009915    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz" event={"ID":"6bdde153-67bf-4195-a214-3e10b2d34889","Type":"ContainerDied","Data":"5ac07b91ed5895246a846260da15d735e59edd09641b605bf5d30f03d8ff92b3"}
Sep 19 12:30:54.016382 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:54.016345    2571 generic.go:356] "Generic (PLEG): container finished" podID="6bdde153-67bf-4195-a214-3e10b2d34889" containerID="aceed43b3f1cf084170a476b98baf6885e4bf08e35b2e889bb50f8af18ee042c" exitCode=0
Sep 19 12:30:54.016770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:54.016402    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz" event={"ID":"6bdde153-67bf-4195-a214-3e10b2d34889","Type":"ContainerDied","Data":"aceed43b3f1cf084170a476b98baf6885e4bf08e35b2e889bb50f8af18ee042c"}
Sep 19 12:30:55.138815 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.138792    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:55.163634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.163608    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2d8wb\" (UniqueName: \"kubernetes.io/projected/6bdde153-67bf-4195-a214-3e10b2d34889-kube-api-access-2d8wb\") pod \"6bdde153-67bf-4195-a214-3e10b2d34889\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") "
Sep 19 12:30:55.163770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.163647    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-util\") pod \"6bdde153-67bf-4195-a214-3e10b2d34889\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") "
Sep 19 12:30:55.163770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.163664    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-bundle\") pod \"6bdde153-67bf-4195-a214-3e10b2d34889\" (UID: \"6bdde153-67bf-4195-a214-3e10b2d34889\") "
Sep 19 12:30:55.164548 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.164519    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-bundle" (OuterVolumeSpecName: "bundle") pod "6bdde153-67bf-4195-a214-3e10b2d34889" (UID: "6bdde153-67bf-4195-a214-3e10b2d34889"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:30:55.165693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.165667    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bdde153-67bf-4195-a214-3e10b2d34889-kube-api-access-2d8wb" (OuterVolumeSpecName: "kube-api-access-2d8wb") pod "6bdde153-67bf-4195-a214-3e10b2d34889" (UID: "6bdde153-67bf-4195-a214-3e10b2d34889"). InnerVolumeSpecName "kube-api-access-2d8wb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:30:55.172543 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.172509    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-util" (OuterVolumeSpecName: "util") pod "6bdde153-67bf-4195-a214-3e10b2d34889" (UID: "6bdde153-67bf-4195-a214-3e10b2d34889"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:30:55.251115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.251077    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:55.264110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.264091    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2d8wb\" (UniqueName: \"kubernetes.io/projected/6bdde153-67bf-4195-a214-3e10b2d34889-kube-api-access-2d8wb\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:30:55.264177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.264112    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-util\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:30:55.264177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.264122    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6bdde153-67bf-4195-a214-3e10b2d34889-bundle\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:30:55.297241 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297175    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"]
Sep 19 12:30:55.297435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297424    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="pull"
Sep 19 12:30:55.297492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297437    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="pull"
Sep 19 12:30:55.297492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297445    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="extract"
Sep 19 12:30:55.297492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297450    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="extract"
Sep 19 12:30:55.297492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297471    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="util"
Sep 19 12:30:55.297492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297477    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="util"
Sep 19 12:30:55.297671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.297528    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bdde153-67bf-4195-a214-3e10b2d34889" containerName="extract"
Sep 19 12:30:55.299500 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.299479    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.302567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.302546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 19 12:30:55.302567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.302552    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 19 12:30:55.302724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.302585    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 19 12:30:55.302724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.302608    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6ms5g\""
Sep 19 12:30:55.302871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.302856    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 19 12:30:55.303038 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.303024    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 19 12:30:55.313938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.313912    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"]
Sep 19 12:30:55.327934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.327911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:30:55.365048 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.365017    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c61c76c5-af38-451f-a0d6-43de4887bb64-cert\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.365048 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.365048    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7fds\" (UniqueName: \"kubernetes.io/projected/c61c76c5-af38-451f-a0d6-43de4887bb64-kube-api-access-m7fds\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.365196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.365068    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c61c76c5-af38-451f-a0d6-43de4887bb64-manager-config\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.365196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.365122    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c61c76c5-af38-451f-a0d6-43de4887bb64-metrics-cert\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.465941 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.465910    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c61c76c5-af38-451f-a0d6-43de4887bb64-metrics-cert\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.466125 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.465961    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c61c76c5-af38-451f-a0d6-43de4887bb64-cert\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.466125 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.466082    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7fds\" (UniqueName: \"kubernetes.io/projected/c61c76c5-af38-451f-a0d6-43de4887bb64-kube-api-access-m7fds\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.466243 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.466124    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c61c76c5-af38-451f-a0d6-43de4887bb64-manager-config\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.466896 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.466874    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/c61c76c5-af38-451f-a0d6-43de4887bb64-manager-config\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.468416 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.468393    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c61c76c5-af38-451f-a0d6-43de4887bb64-metrics-cert\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.468520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.468483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c61c76c5-af38-451f-a0d6-43de4887bb64-cert\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.489775 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.489753    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7fds\" (UniqueName: \"kubernetes.io/projected/c61c76c5-af38-451f-a0d6-43de4887bb64-kube-api-access-m7fds\") pod \"lws-controller-manager-6bdf74d45f-hw9cv\" (UID: \"c61c76c5-af38-451f-a0d6-43de4887bb64\") " pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.608516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.608413    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:30:55.878404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:55.878380    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"]
Sep 19 12:30:55.880490 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:30:55.880443    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc61c76c5_af38_451f_a0d6_43de4887bb64.slice/crio-dce8e9ed22b841f83035b11bf0c522f6fa595574dafdc0a805f7cedc22097586 WatchSource:0}: Error finding container dce8e9ed22b841f83035b11bf0c522f6fa595574dafdc0a805f7cedc22097586: Status 404 returned error can't find the container with id dce8e9ed22b841f83035b11bf0c522f6fa595574dafdc0a805f7cedc22097586
Sep 19 12:30:56.028823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:56.028797    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz"
Sep 19 12:30:56.028823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:56.028803    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afxbdxz" event={"ID":"6bdde153-67bf-4195-a214-3e10b2d34889","Type":"ContainerDied","Data":"aa91b6114821a589d743959dceb514081dbe1b05a831f16a2f4d71804c9da2c6"}
Sep 19 12:30:56.029044 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:56.028874    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa91b6114821a589d743959dceb514081dbe1b05a831f16a2f4d71804c9da2c6"
Sep 19 12:30:56.030008 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:30:56.029989    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv" event={"ID":"c61c76c5-af38-451f-a0d6-43de4887bb64","Type":"ContainerStarted","Data":"dce8e9ed22b841f83035b11bf0c522f6fa595574dafdc0a805f7cedc22097586"}
Sep 19 12:31:02.052859 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:02.052787    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv" event={"ID":"c61c76c5-af38-451f-a0d6-43de4887bb64","Type":"ContainerStarted","Data":"eaf2e4050f66afe79f36b65cb515e0d3c47c3f2bf9d04569f3b3a4ac8299afe1"}
Sep 19 12:31:02.053192 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:02.052896    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:31:02.068758 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:02.068718    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv" podStartSLOduration=1.21161045 podStartE2EDuration="7.068706434s" totalImagesPullingTime="5.857095984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:30:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:31:02.067179001 +0000 UTC m=+345.427421942" watchObservedRunningTime="2026-09-19 12:31:02.068706434 +0000 UTC m=+345.428949374"
Sep 19 12:31:05.178935 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.178899    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"]
Sep 19 12:31:05.182110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.182096    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.184784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.184761    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 19 12:31:05.184907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.184787    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 19 12:31:05.184907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.184802    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2v6sm\""
Sep 19 12:31:05.190590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.190567    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"]
Sep 19 12:31:05.228985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.228958    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s5pj\" (UniqueName: \"kubernetes.io/projected/87068b13-7b39-486e-9765-e4618e20c8af-kube-api-access-8s5pj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.228985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.228986    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.229152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.229008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.329615 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.329582    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s5pj\" (UniqueName: \"kubernetes.io/projected/87068b13-7b39-486e-9765-e4618e20c8af-kube-api-access-8s5pj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.329615 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.329617    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.329819 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.329639    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.330058 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.330038    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.330095 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.330040    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.338877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.338854    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s5pj\" (UniqueName: \"kubernetes.io/projected/87068b13-7b39-486e-9765-e4618e20c8af-kube-api-access-8s5pj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.491828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.491795    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:05.763476 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.763432    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"]
Sep 19 12:31:05.765241 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:31:05.765216    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87068b13_7b39_486e_9765_e4618e20c8af.slice/crio-51f3bd6c7dd97e6bfdc8ea3d8e0b522e610705af0235e7d6f04f3933797d4fda WatchSource:0}: Error finding container 51f3bd6c7dd97e6bfdc8ea3d8e0b522e610705af0235e7d6f04f3933797d4fda: Status 404 returned error can't find the container with id 51f3bd6c7dd97e6bfdc8ea3d8e0b522e610705af0235e7d6f04f3933797d4fda
Sep 19 12:31:05.958494 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:05.958444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:06.062392 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.062304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:06.067194 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.067168    2571 generic.go:356] "Generic (PLEG): container finished" podID="87068b13-7b39-486e-9765-e4618e20c8af" containerID="8c8525898da4cd0711986d7dc087a41d227a711a4b6f20b679776a5d39dc9a9d" exitCode=0
Sep 19 12:31:06.067346 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.067226    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j" event={"ID":"87068b13-7b39-486e-9765-e4618e20c8af","Type":"ContainerDied","Data":"8c8525898da4cd0711986d7dc087a41d227a711a4b6f20b679776a5d39dc9a9d"}
Sep 19 12:31:06.067346 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.067261    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j" event={"ID":"87068b13-7b39-486e-9765-e4618e20c8af","Type":"ContainerStarted","Data":"51f3bd6c7dd97e6bfdc8ea3d8e0b522e610705af0235e7d6f04f3933797d4fda"}
Sep 19 12:31:06.477645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.477610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:06.685781 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.685747    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:06.788426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.788391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:06.860193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.860119    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"]
Sep 19 12:31:06.863289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.863274    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:06.865788 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.865767    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 19 12:31:06.865904 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.865847    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 19 12:31:06.866160 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.866145    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-bj9t6\""
Sep 19 12:31:06.866245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.866148    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 19 12:31:06.866245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.866204    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 19 12:31:06.881309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.881284    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"]
Sep 19 12:31:06.939750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.939712    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-webhook-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:06.939750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.939755    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6mqf\" (UniqueName: \"kubernetes.io/projected/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-kube-api-access-v6mqf\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:06.939950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.939807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-manifests\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:06.939950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.939862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-tmp\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:06.939950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.939885    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:06.969289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:06.969254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:07.040519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.040484    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-tmp\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.040519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.040522    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.040740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.040557    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-webhook-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.040740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.040590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6mqf\" (UniqueName: \"kubernetes.io/projected/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-kube-api-access-v6mqf\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.040740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.040612    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-manifests\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.041006 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.040982    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-manifests\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.042853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.042829    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-tmp\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.043057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.043038    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.043113 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.043065    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-webhook-cert\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.050364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.050340    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6mqf\" (UniqueName: \"kubernetes.io/projected/5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d-kube-api-access-v6mqf\") pod \"opendatahub-operator-controller-manager-774bc4964b-qnwwc\" (UID: \"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d\") " pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.059487 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.059446    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:07.071872 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.071852    2571 generic.go:356] "Generic (PLEG): container finished" podID="87068b13-7b39-486e-9765-e4618e20c8af" containerID="b9c2146d8149824060b3712532ec124dcb4421560958a1959f9ed437f70f96f2" exitCode=0
Sep 19 12:31:07.071938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.071912    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j" event={"ID":"87068b13-7b39-486e-9765-e4618e20c8af","Type":"ContainerDied","Data":"b9c2146d8149824060b3712532ec124dcb4421560958a1959f9ed437f70f96f2"}
Sep 19 12:31:07.161688 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.161660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:07.172584 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.172548    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:07.263040 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.263009    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:07.365504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.365405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:07.437685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.437652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:07.449496 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:07.449474    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"]
Sep 19 12:31:07.451747 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:31:07.451727    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bf1c9fe_a4bd_4cb1_9af9_e2c735d3032d.slice/crio-e7319d0b674938833ff73ccc129aa78bc14c5307654a2faeab67adfc41dec867 WatchSource:0}: Error finding container e7319d0b674938833ff73ccc129aa78bc14c5307654a2faeab67adfc41dec867: Status 404 returned error can't find the container with id e7319d0b674938833ff73ccc129aa78bc14c5307654a2faeab67adfc41dec867
Sep 19 12:31:08.081080 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:08.081046    2571 generic.go:356] "Generic (PLEG): container finished" podID="87068b13-7b39-486e-9765-e4618e20c8af" containerID="d5fe04ebc5fd381d41bcecb9e105aa66a907c3776c0c2a150d0d943543cc855d" exitCode=0
Sep 19 12:31:08.081571 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:08.081201    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j" event={"ID":"87068b13-7b39-486e-9765-e4618e20c8af","Type":"ContainerDied","Data":"d5fe04ebc5fd381d41bcecb9e105aa66a907c3776c0c2a150d0d943543cc855d"}
Sep 19 12:31:08.082954 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:08.082925    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc" event={"ID":"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d","Type":"ContainerStarted","Data":"e7319d0b674938833ff73ccc129aa78bc14c5307654a2faeab67adfc41dec867"}
Sep 19 12:31:10.354894 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.354867    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:10.364073 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.364051    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-bundle\") pod \"87068b13-7b39-486e-9765-e4618e20c8af\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") "
Sep 19 12:31:10.364172 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.364095    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s5pj\" (UniqueName: \"kubernetes.io/projected/87068b13-7b39-486e-9765-e4618e20c8af-kube-api-access-8s5pj\") pod \"87068b13-7b39-486e-9765-e4618e20c8af\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") "
Sep 19 12:31:10.364172 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.364118    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-util\") pod \"87068b13-7b39-486e-9765-e4618e20c8af\" (UID: \"87068b13-7b39-486e-9765-e4618e20c8af\") "
Sep 19 12:31:10.364751 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.364725    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-bundle" (OuterVolumeSpecName: "bundle") pod "87068b13-7b39-486e-9765-e4618e20c8af" (UID: "87068b13-7b39-486e-9765-e4618e20c8af"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:31:10.366197 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.366173    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87068b13-7b39-486e-9765-e4618e20c8af-kube-api-access-8s5pj" (OuterVolumeSpecName: "kube-api-access-8s5pj") pod "87068b13-7b39-486e-9765-e4618e20c8af" (UID: "87068b13-7b39-486e-9765-e4618e20c8af"). InnerVolumeSpecName "kube-api-access-8s5pj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:31:10.369145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.369110    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-util" (OuterVolumeSpecName: "util") pod "87068b13-7b39-486e-9765-e4618e20c8af" (UID: "87068b13-7b39-486e-9765-e4618e20c8af"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:31:10.465017 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.464994    2571 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-bundle\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:31:10.465017 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.465018    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s5pj\" (UniqueName: \"kubernetes.io/projected/87068b13-7b39-486e-9765-e4618e20c8af-kube-api-access-8s5pj\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:31:10.465137 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:10.465028    2571 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/87068b13-7b39-486e-9765-e4618e20c8af-util\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:31:11.098826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:11.098743    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j"
Sep 19 12:31:11.098826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:11.098741    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b0db26j" event={"ID":"87068b13-7b39-486e-9765-e4618e20c8af","Type":"ContainerDied","Data":"51f3bd6c7dd97e6bfdc8ea3d8e0b522e610705af0235e7d6f04f3933797d4fda"}
Sep 19 12:31:11.099046 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:11.098863    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51f3bd6c7dd97e6bfdc8ea3d8e0b522e610705af0235e7d6f04f3933797d4fda"
Sep 19 12:31:11.100404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:11.100380    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc" event={"ID":"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d","Type":"ContainerStarted","Data":"0a603b0c32ba8ef0d7914bab81e3e6fb5324be00fab2ac3d030eaf689cee4726"}
Sep 19 12:31:12.105967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:12.105927    2571 generic.go:356] "Generic (PLEG): container finished" podID="5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d" containerID="0a603b0c32ba8ef0d7914bab81e3e6fb5324be00fab2ac3d030eaf689cee4726" exitCode=0
Sep 19 12:31:12.106348 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:12.106014    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc" event={"ID":"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d","Type":"ContainerDied","Data":"0a603b0c32ba8ef0d7914bab81e3e6fb5324be00fab2ac3d030eaf689cee4726"}
Sep 19 12:31:13.058549 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:13.058518    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6bdf74d45f-hw9cv"
Sep 19 12:31:13.111973 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:13.111942    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc" event={"ID":"5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d","Type":"ContainerStarted","Data":"96ee0bc19f270cdf9049af89836615b1f7bf809cf996c09ccd0f6565f6ba1547"}
Sep 19 12:31:13.112408 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:13.112012    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:13.133044 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:13.132973    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc" podStartSLOduration=2.770789078 podStartE2EDuration="7.132957619s" totalImagesPullingTime="3.279659512s" totalInitContainerRuntime="1.082509029s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:31:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:31:13.131886244 +0000 UTC m=+356.492129198" watchObservedRunningTime="2026-09-19 12:31:13.132957619 +0000 UTC m=+356.493200559"
Sep 19 12:31:24.118579 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:24.118544    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-774bc4964b-qnwwc"
Sep 19 12:31:25.977541 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:25.977504    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:26.095874 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.095837    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:26.317422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.317340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:26.415940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.415910    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:26.448482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448435    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"]
Sep 19 12:31:26.448707 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448695    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="util"
Sep 19 12:31:26.448754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448708    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="util"
Sep 19 12:31:26.448754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448721    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="pull"
Sep 19 12:31:26.448754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448726    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="pull"
Sep 19 12:31:26.448754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448735    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="extract"
Sep 19 12:31:26.448754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448741    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="extract"
Sep 19 12:31:26.448920 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.448783    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="87068b13-7b39-486e-9765-e4618e20c8af" containerName="extract"
Sep 19 12:31:26.602091 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.602012    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"]
Sep 19 12:31:26.602257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.602160    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.605018 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.604995    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 19 12:31:26.605142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.605017    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 19 12:31:26.606029 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.606004    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9gb6c\""
Sep 19 12:31:26.606152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.606048    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 19 12:31:26.606152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.606107    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 19 12:31:26.686129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.686097    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18da225e-0f9e-4994-bf20-e2a4d9871678-tmp\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.686305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.686143    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nrsc\" (UniqueName: \"kubernetes.io/projected/18da225e-0f9e-4994-bf20-e2a4d9871678-kube-api-access-7nrsc\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.686305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.686232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18da225e-0f9e-4994-bf20-e2a4d9871678-tls-certs\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.787054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.787019    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18da225e-0f9e-4994-bf20-e2a4d9871678-tls-certs\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.787255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.787064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18da225e-0f9e-4994-bf20-e2a4d9871678-tmp\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.787255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.787089    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nrsc\" (UniqueName: \"kubernetes.io/projected/18da225e-0f9e-4994-bf20-e2a4d9871678-kube-api-access-7nrsc\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.789278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.789256    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18da225e-0f9e-4994-bf20-e2a4d9871678-tmp\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.789599 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.789581    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18da225e-0f9e-4994-bf20-e2a4d9871678-tls-certs\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.794415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.794397    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nrsc\" (UniqueName: \"kubernetes.io/projected/18da225e-0f9e-4994-bf20-e2a4d9871678-kube-api-access-7nrsc\") pod \"kube-auth-proxy-5c9bc57f57-wn4c7\" (UID: \"18da225e-0f9e-4994-bf20-e2a4d9871678\") " pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:26.912507 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:26.912440    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"
Sep 19 12:31:27.208074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:27.208049    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7"]
Sep 19 12:31:27.210432 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:31:27.210406    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18da225e_0f9e_4994_bf20_e2a4d9871678.slice/crio-c4f995f69975a7978e48b327be5f6de6793eaa7d47228d75d48e057abe3832d4 WatchSource:0}: Error finding container c4f995f69975a7978e48b327be5f6de6793eaa7d47228d75d48e057abe3832d4: Status 404 returned error can't find the container with id c4f995f69975a7978e48b327be5f6de6793eaa7d47228d75d48e057abe3832d4
Sep 19 12:31:27.701809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:27.701779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:27.821587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:27.821539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:28.174977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:28.174890    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7" event={"ID":"18da225e-0f9e-4994-bf20-e2a4d9871678","Type":"ContainerStarted","Data":"c4f995f69975a7978e48b327be5f6de6793eaa7d47228d75d48e057abe3832d4"}
Sep 19 12:31:28.971211 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:28.971177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:30.185871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:30.185840    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7" event={"ID":"18da225e-0f9e-4994-bf20-e2a4d9871678","Type":"ContainerStarted","Data":"012536f83ca5c3ff0804f48bb5e0a97547dfbd8fbcbb18b8094af3f9357e5d5e"}
Sep 19 12:31:30.206571 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:30.206533    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5c9bc57f57-wn4c7" podStartSLOduration=1.478146617 podStartE2EDuration="4.206520636s" totalImagesPullingTime="2.728374019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:31:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:31:30.204495547 +0000 UTC m=+373.564738487" watchObservedRunningTime="2026-09-19 12:31:30.206520636 +0000 UTC m=+373.566763576"
Sep 19 12:31:33.295995 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:33.295959    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:33.359380 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:33.359335    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:33.860413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:33.860377    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:33.959984 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:33.959949    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:34.061633 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:34.061596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:36.982023 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:36.981982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:37.084843 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:37.084801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:37.182684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:37.182649    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:37.412359 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:37.412270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:37.510866 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:37.510836    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:45.493129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:45.493097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:45.616868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:45.616824    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:47.227516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:47.227475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:47.424610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:47.424567    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:47.597274 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:47.597178    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:47.920216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:47.920125    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:48.606862 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:48.606832    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:48.711036 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:48.711002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:48.813120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:48.813077    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:56.820327 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:56.820276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:56.922158 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:56.922122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:31:56.985277 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:31:56.985233    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:38.196409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:38.196310    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:38.301842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:38.301794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:53.284080 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:53.284044    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:54.088421 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:54.088384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:54.956114 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:54.956078    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:57.195631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:57.195599    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:32:58.485750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:32:58.485715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:00.322940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:00.322908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:00.495422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:00.495380    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:02.276959 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:02.276928    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:02.519495 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:02.519434    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:02.695248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:02.695153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:02.801133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:02.801100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:03.981919 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:03.981885    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:06.597752 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.597718    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"]
Sep 19 12:33:06.600736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.600710    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.601397 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.601361    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:06.603623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.603603    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v24kx\""
Sep 19 12:33:06.603868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.603854    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 19 12:33:06.603938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.603907    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 19 12:33:06.604799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.604779    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 19 12:33:06.604921 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.604802    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 19 12:33:06.610146 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.610117    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"]
Sep 19 12:33:06.752139 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.752098    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.752139 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.752140    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffwcb\" (UniqueName: \"kubernetes.io/projected/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-kube-api-access-ffwcb\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.752338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.752234    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.853516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.853404    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.853516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.853502    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.853730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.853527    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffwcb\" (UniqueName: \"kubernetes.io/projected/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-kube-api-access-ffwcb\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.853730 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:33:06.853637    2571 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 19 12:33:06.853730 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:33:06.853707    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-plugin-serving-cert podName:cd9735f6-5b7a-47bc-bdc3-954c96ed7508 nodeName:}" failed. No retries permitted until 2026-09-19 12:33:07.353691632 +0000 UTC m=+470.713934552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-2wtrl" (UID: "cd9735f6-5b7a-47bc-bdc3-954c96ed7508") : secret "plugin-serving-cert" not found
Sep 19 12:33:06.854164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.854139    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:06.864384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:06.864352    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffwcb\" (UniqueName: \"kubernetes.io/projected/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-kube-api-access-ffwcb\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:07.014651 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:07.014613    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:07.357269 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:07.357227    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:07.359660 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:07.359639    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/cd9735f6-5b7a-47bc-bdc3-954c96ed7508-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-2wtrl\" (UID: \"cd9735f6-5b7a-47bc-bdc3-954c96ed7508\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:07.510690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:07.510649    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"
Sep 19 12:33:07.783022 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:07.782997    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl"]
Sep 19 12:33:07.785421 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:33:07.785396    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcd9735f6_5b7a_47bc_bdc3_954c96ed7508.slice/crio-fbb6c6f45dff4948f5884fd6b3e246583846740a9b4a7f009ddca4538c449e71 WatchSource:0}: Error finding container fbb6c6f45dff4948f5884fd6b3e246583846740a9b4a7f009ddca4538c449e71: Status 404 returned error can't find the container with id fbb6c6f45dff4948f5884fd6b3e246583846740a9b4a7f009ddca4538c449e71
Sep 19 12:33:08.283016 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:08.282981    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:08.384774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:08.384739    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:08.597808 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:08.597694    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl" event={"ID":"cd9735f6-5b7a-47bc-bdc3-954c96ed7508","Type":"ContainerStarted","Data":"fbb6c6f45dff4948f5884fd6b3e246583846740a9b4a7f009ddca4538c449e71"}
Sep 19 12:33:08.685335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:08.685297    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:08.733810 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:08.733774    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:09.138387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:09.138342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:09.233789 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:09.233733    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:12.618669 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:12.618576    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl" event={"ID":"cd9735f6-5b7a-47bc-bdc3-954c96ed7508","Type":"ContainerStarted","Data":"f568b0685c1a11e600672aa6a91a1ede2fb640e67bbd3fab57da09820f903eac"}
Sep 19 12:33:12.635116 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:12.635070    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-2wtrl" podStartSLOduration=2.183490729 podStartE2EDuration="6.63505644s" totalImagesPullingTime="4.451565711s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:33:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:33:12.634571383 +0000 UTC m=+475.994814325" watchObservedRunningTime="2026-09-19 12:33:12.63505644 +0000 UTC m=+475.995299380"
Sep 19 12:33:15.306234 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:15.306195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:15.403273 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:15.403231    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:17.900217 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:17.900179    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:17.989512 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:17.989478    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:24.307785 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:24.307750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:24.409482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:24.409429    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:25.552187 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:25.552154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:25.652835 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:25.652799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:27.400020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:27.399982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:27.495279 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:27.495229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:29.512923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:29.512888    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:29.607272 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:29.607233    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:29.908078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:29.907989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:30.011685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:30.011650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:30.112287 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:30.112248    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:30.217511 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:30.217475    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:37.111184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:37.111149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:37.212556 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:37.212522    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:39.550905 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:39.550864    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:39.651533 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:39.651501    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:45.571021 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:45.570967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:46.110724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:46.110684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:46.209325 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:46.209292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:49.034181 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:49.034153    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:49.082946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:49.082902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:57.809301 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:57.809258    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:57.910546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:57.910508    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:58.008861 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.008823    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:58.189853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.189776    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:33:58.210857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.210832    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:33:58.211008 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.210927    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.213701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.213672    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 19 12:33:58.215556 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.215536    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:33:58.247814 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.247779    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/04627672-f1c7-4854-a09a-0b8217e717de-config-file\") pod \"limitador-limitador-84994f7ffc-gttp7\" (UID: \"04627672-f1c7-4854-a09a-0b8217e717de\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.247978 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.247854    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctssq\" (UniqueName: \"kubernetes.io/projected/04627672-f1c7-4854-a09a-0b8217e717de-kube-api-access-ctssq\") pod \"limitador-limitador-84994f7ffc-gttp7\" (UID: \"04627672-f1c7-4854-a09a-0b8217e717de\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.349093 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.349060    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctssq\" (UniqueName: \"kubernetes.io/projected/04627672-f1c7-4854-a09a-0b8217e717de-kube-api-access-ctssq\") pod \"limitador-limitador-84994f7ffc-gttp7\" (UID: \"04627672-f1c7-4854-a09a-0b8217e717de\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.349093 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.349097    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/04627672-f1c7-4854-a09a-0b8217e717de-config-file\") pod \"limitador-limitador-84994f7ffc-gttp7\" (UID: \"04627672-f1c7-4854-a09a-0b8217e717de\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.349688 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.349673    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/04627672-f1c7-4854-a09a-0b8217e717de-config-file\") pod \"limitador-limitador-84994f7ffc-gttp7\" (UID: \"04627672-f1c7-4854-a09a-0b8217e717de\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.356550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.356527    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctssq\" (UniqueName: \"kubernetes.io/projected/04627672-f1c7-4854-a09a-0b8217e717de-kube-api-access-ctssq\") pod \"limitador-limitador-84994f7ffc-gttp7\" (UID: \"04627672-f1c7-4854-a09a-0b8217e717de\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.520592 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.520560    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:33:58.637759 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.637728    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:58.716685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.716650    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-zg6xx"]
Sep 19 12:33:58.720253 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.720229    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:33:58.720612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.720594    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:33:58.723074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.723052    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ll9kr\""
Sep 19 12:33:58.727216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.727187    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zg6xx"]
Sep 19 12:33:58.791930 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.791907    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:33:58.794413 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:33:58.794380    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04627672_f1c7_4854_a09a_0b8217e717de.slice/crio-0ebdfc9e96a8a4d14bc982b858c2277245d4eab19b3b75c4820d3e6cbb33a77e WatchSource:0}: Error finding container 0ebdfc9e96a8a4d14bc982b858c2277245d4eab19b3b75c4820d3e6cbb33a77e: Status 404 returned error can't find the container with id 0ebdfc9e96a8a4d14bc982b858c2277245d4eab19b3b75c4820d3e6cbb33a77e
Sep 19 12:33:58.805560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.805528    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7" event={"ID":"04627672-f1c7-4854-a09a-0b8217e717de","Type":"ContainerStarted","Data":"0ebdfc9e96a8a4d14bc982b858c2277245d4eab19b3b75c4820d3e6cbb33a77e"}
Sep 19 12:33:58.852663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.852629    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhvdg\" (UniqueName: \"kubernetes.io/projected/c5e209ac-b702-4921-ab21-24cbee6456d0-kube-api-access-qhvdg\") pod \"authorino-59c974bd48-zg6xx\" (UID: \"c5e209ac-b702-4921-ab21-24cbee6456d0\") " pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:33:58.953025 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.952996    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvdg\" (UniqueName: \"kubernetes.io/projected/c5e209ac-b702-4921-ab21-24cbee6456d0-kube-api-access-qhvdg\") pod \"authorino-59c974bd48-zg6xx\" (UID: \"c5e209ac-b702-4921-ab21-24cbee6456d0\") " pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:33:58.962631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:58.962609    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvdg\" (UniqueName: \"kubernetes.io/projected/c5e209ac-b702-4921-ab21-24cbee6456d0-kube-api-access-qhvdg\") pod \"authorino-59c974bd48-zg6xx\" (UID: \"c5e209ac-b702-4921-ab21-24cbee6456d0\") " pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:33:59.030694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:59.030664    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:33:59.297122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:59.297057    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zg6xx"]
Sep 19 12:33:59.299085 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:33:59.299060    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc5e209ac_b702_4921_ab21_24cbee6456d0.slice/crio-989f2370e020b5b39c12ec00dfdd1b82dfe07936909379a65f700ec92595c0f6 WatchSource:0}: Error finding container 989f2370e020b5b39c12ec00dfdd1b82dfe07936909379a65f700ec92595c0f6: Status 404 returned error can't find the container with id 989f2370e020b5b39c12ec00dfdd1b82dfe07936909379a65f700ec92595c0f6
Sep 19 12:33:59.811273 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:33:59.811235    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zg6xx" event={"ID":"c5e209ac-b702-4921-ab21-24cbee6456d0","Type":"ContainerStarted","Data":"989f2370e020b5b39c12ec00dfdd1b82dfe07936909379a65f700ec92595c0f6"}
Sep 19 12:34:01.821533 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:01.821421    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7" event={"ID":"04627672-f1c7-4854-a09a-0b8217e717de","Type":"ContainerStarted","Data":"a9074ec9ec48d1782b19e7138296da55a969254ea2b383b8930250f892679ac5"}
Sep 19 12:34:01.821914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:01.821539    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:34:01.822929 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:01.822905    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zg6xx" event={"ID":"c5e209ac-b702-4921-ab21-24cbee6456d0","Type":"ContainerStarted","Data":"201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc"}
Sep 19 12:34:01.838619 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:01.838572    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7" podStartSLOduration=1.146395345 podStartE2EDuration="3.83855884s" totalImagesPullingTime="2.692163495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:33:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:34:01.836962135 +0000 UTC m=+525.197205076" watchObservedRunningTime="2026-09-19 12:34:01.83855884 +0000 UTC m=+525.198801785"
Sep 19 12:34:01.851363 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:01.851322    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-zg6xx" podStartSLOduration=1.657851345 podStartE2EDuration="3.851310803s" totalImagesPullingTime="2.193459458s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:33:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:34:01.850682025 +0000 UTC m=+525.210924965" watchObservedRunningTime="2026-09-19 12:34:01.851310803 +0000 UTC m=+525.211553747"
Sep 19 12:34:12.828240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:12.828208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-gttp7"
Sep 19 12:34:14.030454 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:14.030413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:14.130843 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:14.130797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:14.487624 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:14.487590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:17.179986 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:17.179950    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:26.931331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:26.931296    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:27.218206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.218175    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-wn89t"]
Sep 19 12:34:27.221612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.221597    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:27.229770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.229748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-wn89t"]
Sep 19 12:34:27.239767 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.239738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:27.281589 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.281571    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wn89t"]
Sep 19 12:34:27.281742 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:34:27.281726    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-4qs7s], unattached volumes=[], failed to process volumes=[kube-api-access-4qs7s]: context canceled" pod="kuadrant-system/authorino-557f876f69-wn89t" podUID="5ac14024-be4f-41c5-9562-d31fb8bf6039"
Sep 19 12:34:27.308884 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.308861    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-6bfmt"]
Sep 19 12:34:27.311868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.311851    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.314444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.314428    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 19 12:34:27.320309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.320289    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-6bfmt"]
Sep 19 12:34:27.381842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.381818    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qs7s\" (UniqueName: \"kubernetes.io/projected/5ac14024-be4f-41c5-9562-d31fb8bf6039-kube-api-access-4qs7s\") pod \"authorino-557f876f69-wn89t\" (UID: \"5ac14024-be4f-41c5-9562-d31fb8bf6039\") " pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:27.404236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.404211    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-6bfmt"]
Sep 19 12:34:27.404390 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:34:27.404373    2571 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7px8n tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-6bfmt" podUID="1f593a40-9c87-4b85-96dc-ee259b55df8c"
Sep 19 12:34:27.482970 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.482915    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qs7s\" (UniqueName: \"kubernetes.io/projected/5ac14024-be4f-41c5-9562-d31fb8bf6039-kube-api-access-4qs7s\") pod \"authorino-557f876f69-wn89t\" (UID: \"5ac14024-be4f-41c5-9562-d31fb8bf6039\") " pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:27.482970 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.482950    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f593a40-9c87-4b85-96dc-ee259b55df8c-tls-cert\") pod \"authorino-6d4dfc649f-6bfmt\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") " pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.483089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.482984    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7px8n\" (UniqueName: \"kubernetes.io/projected/1f593a40-9c87-4b85-96dc-ee259b55df8c-kube-api-access-7px8n\") pod \"authorino-6d4dfc649f-6bfmt\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") " pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.490467 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.490442    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qs7s\" (UniqueName: \"kubernetes.io/projected/5ac14024-be4f-41c5-9562-d31fb8bf6039-kube-api-access-4qs7s\") pod \"authorino-557f876f69-wn89t\" (UID: \"5ac14024-be4f-41c5-9562-d31fb8bf6039\") " pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:27.584001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.583978    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7px8n\" (UniqueName: \"kubernetes.io/projected/1f593a40-9c87-4b85-96dc-ee259b55df8c-kube-api-access-7px8n\") pod \"authorino-6d4dfc649f-6bfmt\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") " pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.584091 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.584027    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f593a40-9c87-4b85-96dc-ee259b55df8c-tls-cert\") pod \"authorino-6d4dfc649f-6bfmt\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") " pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.586195 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.586179    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f593a40-9c87-4b85-96dc-ee259b55df8c-tls-cert\") pod \"authorino-6d4dfc649f-6bfmt\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") " pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.591736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.591715    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7px8n\" (UniqueName: \"kubernetes.io/projected/1f593a40-9c87-4b85-96dc-ee259b55df8c-kube-api-access-7px8n\") pod \"authorino-6d4dfc649f-6bfmt\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") " pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.929349 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.929290    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:27.929452 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.929294    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:27.936009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.935987    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:27.938699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:27.938683    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:28.086994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.086967    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qs7s\" (UniqueName: \"kubernetes.io/projected/5ac14024-be4f-41c5-9562-d31fb8bf6039-kube-api-access-4qs7s\") pod \"5ac14024-be4f-41c5-9562-d31fb8bf6039\" (UID: \"5ac14024-be4f-41c5-9562-d31fb8bf6039\") "
Sep 19 12:34:28.087141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.087002    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7px8n\" (UniqueName: \"kubernetes.io/projected/1f593a40-9c87-4b85-96dc-ee259b55df8c-kube-api-access-7px8n\") pod \"1f593a40-9c87-4b85-96dc-ee259b55df8c\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") "
Sep 19 12:34:28.087141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.087043    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f593a40-9c87-4b85-96dc-ee259b55df8c-tls-cert\") pod \"1f593a40-9c87-4b85-96dc-ee259b55df8c\" (UID: \"1f593a40-9c87-4b85-96dc-ee259b55df8c\") "
Sep 19 12:34:28.089244 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.089217    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f593a40-9c87-4b85-96dc-ee259b55df8c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1f593a40-9c87-4b85-96dc-ee259b55df8c" (UID: "1f593a40-9c87-4b85-96dc-ee259b55df8c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:34:28.089244 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.089232    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ac14024-be4f-41c5-9562-d31fb8bf6039-kube-api-access-4qs7s" (OuterVolumeSpecName: "kube-api-access-4qs7s") pod "5ac14024-be4f-41c5-9562-d31fb8bf6039" (UID: "5ac14024-be4f-41c5-9562-d31fb8bf6039"). InnerVolumeSpecName "kube-api-access-4qs7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:34:28.089364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.089226    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f593a40-9c87-4b85-96dc-ee259b55df8c-kube-api-access-7px8n" (OuterVolumeSpecName: "kube-api-access-7px8n") pod "1f593a40-9c87-4b85-96dc-ee259b55df8c" (UID: "1f593a40-9c87-4b85-96dc-ee259b55df8c"). InnerVolumeSpecName "kube-api-access-7px8n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:34:28.187425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.187373    2571 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1f593a40-9c87-4b85-96dc-ee259b55df8c-tls-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:34:28.187425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.187392    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qs7s\" (UniqueName: \"kubernetes.io/projected/5ac14024-be4f-41c5-9562-d31fb8bf6039-kube-api-access-4qs7s\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:34:28.187425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.187404    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7px8n\" (UniqueName: \"kubernetes.io/projected/1f593a40-9c87-4b85-96dc-ee259b55df8c-kube-api-access-7px8n\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:34:28.686096 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.686060    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zg6xx"]
Sep 19 12:34:28.686353 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.686324    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-zg6xx" podUID="c5e209ac-b702-4921-ab21-24cbee6456d0" containerName="authorino" containerID="cri-o://201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc" gracePeriod=30
Sep 19 12:34:28.931456 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.931431    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:34:28.935046 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935024    2571 generic.go:356] "Generic (PLEG): container finished" podID="c5e209ac-b702-4921-ab21-24cbee6456d0" containerID="201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc" exitCode=0
Sep 19 12:34:28.935134 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935058    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-zg6xx"
Sep 19 12:34:28.935134 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935108    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zg6xx" event={"ID":"c5e209ac-b702-4921-ab21-24cbee6456d0","Type":"ContainerDied","Data":"201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc"}
Sep 19 12:34:28.935212 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935145    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-zg6xx" event={"ID":"c5e209ac-b702-4921-ab21-24cbee6456d0","Type":"ContainerDied","Data":"989f2370e020b5b39c12ec00dfdd1b82dfe07936909379a65f700ec92595c0f6"}
Sep 19 12:34:28.935212 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935165    2571 scope.go:117] "RemoveContainer" containerID="201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc"
Sep 19 12:34:28.935212 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935209    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-6bfmt"
Sep 19 12:34:28.935336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.935232    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-wn89t"
Sep 19 12:34:28.942675 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.942658    2571 scope.go:117] "RemoveContainer" containerID="201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc"
Sep 19 12:34:28.942947 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:34:28.942929    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc\": container with ID starting with 201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc not found: ID does not exist" containerID="201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc"
Sep 19 12:34:28.942987 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.942957    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc"} err="failed to get container status \"201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc\": rpc error: code = NotFound desc = could not find container \"201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc\": container with ID starting with 201816c6953db65f6df3d153054def82b219f27b7e9567d8804028b30c9446bc not found: ID does not exist"
Sep 19 12:34:28.973557 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.973533    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-wn89t"]
Sep 19 12:34:28.976585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.976563    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-wn89t"]
Sep 19 12:34:28.995372 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:28.995351    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-6bfmt"]
Sep 19 12:34:29.000848 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:29.000829    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-6bfmt"]
Sep 19 12:34:29.093028 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:29.093007    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhvdg\" (UniqueName: \"kubernetes.io/projected/c5e209ac-b702-4921-ab21-24cbee6456d0-kube-api-access-qhvdg\") pod \"c5e209ac-b702-4921-ab21-24cbee6456d0\" (UID: \"c5e209ac-b702-4921-ab21-24cbee6456d0\") "
Sep 19 12:34:29.095138 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:29.095110    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5e209ac-b702-4921-ab21-24cbee6456d0-kube-api-access-qhvdg" (OuterVolumeSpecName: "kube-api-access-qhvdg") pod "c5e209ac-b702-4921-ab21-24cbee6456d0" (UID: "c5e209ac-b702-4921-ab21-24cbee6456d0"). InnerVolumeSpecName "kube-api-access-qhvdg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:34:29.193423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:29.193373    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhvdg\" (UniqueName: \"kubernetes.io/projected/c5e209ac-b702-4921-ab21-24cbee6456d0-kube-api-access-qhvdg\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:34:29.259937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:29.259914    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zg6xx"]
Sep 19 12:34:29.265988 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:29.265970    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-zg6xx"]
Sep 19 12:34:30.743703 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:30.743670    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f593a40-9c87-4b85-96dc-ee259b55df8c" path="/var/lib/kubelet/pods/1f593a40-9c87-4b85-96dc-ee259b55df8c/volumes"
Sep 19 12:34:30.744059 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:30.743867    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ac14024-be4f-41c5-9562-d31fb8bf6039" path="/var/lib/kubelet/pods/5ac14024-be4f-41c5-9562-d31fb8bf6039/volumes"
Sep 19 12:34:30.744059 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:30.744039    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5e209ac-b702-4921-ab21-24cbee6456d0" path="/var/lib/kubelet/pods/c5e209ac-b702-4921-ab21-24cbee6456d0/volumes"
Sep 19 12:34:33.238746 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:33.238712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:33.303895 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:33.303857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:33.357799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:33.357761    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:33.442864 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:33.442831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:33.495803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:33.495723    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:36.852737 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:36.852704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:36.955216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:36.955189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:34:59.390408 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:34:59.390358    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:06.325319 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:06.325284    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:06.431235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:06.431199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:07.777153 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:07.777114    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:07.846190 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:07.846149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:07.969583 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:07.969538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:08.261343 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:08.261314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:08.361580 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:08.361547    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:08.440894 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:08.440863    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:35:17.898923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:17.898877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:18.858471 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:18.858417    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:24.730072 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.730040    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"]
Sep 19 12:35:24.732592 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.730512    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5e209ac-b702-4921-ab21-24cbee6456d0" containerName="authorino"
Sep 19 12:35:24.732592 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.730528    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5e209ac-b702-4921-ab21-24cbee6456d0" containerName="authorino"
Sep 19 12:35:24.732592 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.730589    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5e209ac-b702-4921-ab21-24cbee6456d0" containerName="authorino"
Sep 19 12:35:24.733415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.733395    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.737799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.737777    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 19 12:35:24.737923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.737818    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 19 12:35:24.738077 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.738059    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pr82q\""
Sep 19 12:35:24.738200 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.738058    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 19 12:35:24.743513 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.743452    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"]
Sep 19 12:35:24.765912 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.765881    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:24.787428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.787400    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d78cb66f-3eca-485f-9587-7e774740a06a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.787563 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.787443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.787563 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.787480    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.787563 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.787496    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wz6h7\" (UniqueName: \"kubernetes.io/projected/d78cb66f-3eca-485f-9587-7e774740a06a-kube-api-access-wz6h7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.787563 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.787519    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.787563 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.787556    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888028 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.887997    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d78cb66f-3eca-485f-9587-7e774740a06a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888052    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888079    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wz6h7\" (UniqueName: \"kubernetes.io/projected/d78cb66f-3eca-485f-9587-7e774740a06a-kube-api-access-wz6h7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888126    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888566 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888543    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888626 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888582    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.888699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.888678    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.890523 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.890500    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d78cb66f-3eca-485f-9587-7e774740a06a-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.890620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.890566    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d78cb66f-3eca-485f-9587-7e774740a06a-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:24.896703 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:24.896659    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wz6h7\" (UniqueName: \"kubernetes.io/projected/d78cb66f-3eca-485f-9587-7e774740a06a-kube-api-access-wz6h7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn\" (UID: \"d78cb66f-3eca-485f-9587-7e774740a06a\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:25.045704 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:25.045591    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:25.315410 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:25.315387    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"]
Sep 19 12:35:25.317741 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:35:25.317713    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78cb66f_3eca_485f_9587_7e774740a06a.slice/crio-39af622a0664b62ec447bc4dc79da4df2ca41d7408835d2e117d423597751423 WatchSource:0}: Error finding container 39af622a0664b62ec447bc4dc79da4df2ca41d7408835d2e117d423597751423: Status 404 returned error can't find the container with id 39af622a0664b62ec447bc4dc79da4df2ca41d7408835d2e117d423597751423
Sep 19 12:35:25.319522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:25.319505    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:35:25.421362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:25.421330    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:35:26.176414 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:26.176379    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn" event={"ID":"d78cb66f-3eca-485f-9587-7e774740a06a","Type":"ContainerStarted","Data":"39af622a0664b62ec447bc4dc79da4df2ca41d7408835d2e117d423597751423"}
Sep 19 12:35:27.181529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:27.181498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn" event={"ID":"d78cb66f-3eca-485f-9587-7e774740a06a","Type":"ContainerStarted","Data":"9e70e3cd4ccfe20c632ee72e591eaf54672f10bff179501369a231715b34840f"}
Sep 19 12:35:27.198268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:27.198225    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn" podStartSLOduration=2.105579239 podStartE2EDuration="3.198210815s" totalImagesPullingTime="1.092631576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:35:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:35:27.197103864 +0000 UTC m=+610.557346805" watchObservedRunningTime="2026-09-19 12:35:27.198210815 +0000 UTC m=+610.558453756"
Sep 19 12:35:35.046059 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:35.045976    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:35.046059 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:35.046045    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:35.058532 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:35.058508    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:35.224448 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:35.224421    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn"
Sep 19 12:35:36.181384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:36.181347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:36.919993 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:36.919957    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:35:41.219968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:41.219930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:42.028300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:42.028259    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:35:46.327445 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.324200    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"]
Sep 19 12:35:46.328411 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.328389    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.331001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.330971    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 19 12:35:46.334436 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.334416    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"]
Sep 19 12:35:46.346426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.346397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:46.458079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.458024    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbd4n\" (UniqueName: \"kubernetes.io/projected/634f3bc1-f890-492c-b2a5-fb76a2792522-kube-api-access-fbd4n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.458269 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.458135    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.458269 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.458171    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.458269 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.458213    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/634f3bc1-f890-492c-b2a5-fb76a2792522-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.458269 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.458235    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.458398 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.458278    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559355    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559410    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbd4n\" (UniqueName: \"kubernetes.io/projected/634f3bc1-f890-492c-b2a5-fb76a2792522-kube-api-access-fbd4n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559433    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559454    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559504    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/634f3bc1-f890-492c-b2a5-fb76a2792522-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559897 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559870    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.559964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559904    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.560013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.559960    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.561791 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.561773    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/634f3bc1-f890-492c-b2a5-fb76a2792522-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.561955 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.561937    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/634f3bc1-f890-492c-b2a5-fb76a2792522-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.567012 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.566995    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbd4n\" (UniqueName: \"kubernetes.io/projected/634f3bc1-f890-492c-b2a5-fb76a2792522-kube-api-access-fbd4n\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hb6dk\" (UID: \"634f3bc1-f890-492c-b2a5-fb76a2792522\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.639042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.638953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:46.906340 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:46.906316    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"]
Sep 19 12:35:46.907858 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:35:46.907830    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634f3bc1_f890_492c_b2a5_fb76a2792522.slice/crio-f58189b9810ed1180cadc060ffaf96afee5f63b87964a695ba904d1c86d152a1 WatchSource:0}: Error finding container f58189b9810ed1180cadc060ffaf96afee5f63b87964a695ba904d1c86d152a1: Status 404 returned error can't find the container with id f58189b9810ed1180cadc060ffaf96afee5f63b87964a695ba904d1c86d152a1
Sep 19 12:35:47.264409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:47.264377    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk" event={"ID":"634f3bc1-f890-492c-b2a5-fb76a2792522","Type":"ContainerStarted","Data":"92aba4d914efc3eb49af3100864be9e10fe44a9dbd1c87892e0d89fcff8406c1"}
Sep 19 12:35:47.264609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:47.264431    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk" event={"ID":"634f3bc1-f890-492c-b2a5-fb76a2792522","Type":"ContainerStarted","Data":"f58189b9810ed1180cadc060ffaf96afee5f63b87964a695ba904d1c86d152a1"}
Sep 19 12:35:47.283904 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:47.283856    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk" podStartSLOduration=1.121455689 podStartE2EDuration="1.283842772s" totalImagesPullingTime="162.387083ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:35:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:35:47.283136736 +0000 UTC m=+630.643379674" watchObservedRunningTime="2026-09-19 12:35:47.283842772 +0000 UTC m=+630.644085714"
Sep 19 12:35:47.522522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:47.522433    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:35:56.639148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:56.639111    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:56.639571 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:56.639213    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:56.651071 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:56.651053    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:57.316756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:57.316724    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hb6dk"
Sep 19 12:35:58.328187 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:58.328149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:35:59.730913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:35:59.730882    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:36:16.721307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.721275    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"]
Sep 19 12:36:16.724628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.724603    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.727308 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.727290    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 19 12:36:16.740145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.740120    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"]
Sep 19 12:36:16.770690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.770651    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:36:16.804232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.804201    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.804232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.804231    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.804423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.804250    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7qjb\" (UniqueName: \"kubernetes.io/projected/91852cea-879e-437d-bc83-d508c22e7806-kube-api-access-n7qjb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.804423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.804403    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.804528 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.804434    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91852cea-879e-437d-bc83-d508c22e7806-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.804563 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.804540    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.904923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.904886    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.904923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.904928    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91852cea-879e-437d-bc83-d508c22e7806-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905044    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905102    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905211 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905125    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905211 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905146    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7qjb\" (UniqueName: \"kubernetes.io/projected/91852cea-879e-437d-bc83-d508c22e7806-kube-api-access-n7qjb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905290    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905417 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905396    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.905506 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.905424    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.907287 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.907266    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91852cea-879e-437d-bc83-d508c22e7806-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.907531 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.907516    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91852cea-879e-437d-bc83-d508c22e7806-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:16.912802 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:16.912779    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7qjb\" (UniqueName: \"kubernetes.io/projected/91852cea-879e-437d-bc83-d508c22e7806-kube-api-access-n7qjb\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-l6hl2\" (UID: \"91852cea-879e-437d-bc83-d508c22e7806\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:17.037050 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:17.036966    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:17.308089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:17.308065    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"]
Sep 19 12:36:17.310189 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:36:17.310142    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91852cea_879e_437d_bc83_d508c22e7806.slice/crio-9ab45cf948a938e77182c2a307d3991001c615b02448cd1a44cc0ce0a90e9513 WatchSource:0}: Error finding container 9ab45cf948a938e77182c2a307d3991001c615b02448cd1a44cc0ce0a90e9513: Status 404 returned error can't find the container with id 9ab45cf948a938e77182c2a307d3991001c615b02448cd1a44cc0ce0a90e9513
Sep 19 12:36:17.399575 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:17.399541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2" event={"ID":"91852cea-879e-437d-bc83-d508c22e7806","Type":"ContainerStarted","Data":"9ab45cf948a938e77182c2a307d3991001c615b02448cd1a44cc0ce0a90e9513"}
Sep 19 12:36:18.405552 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:18.405518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2" event={"ID":"91852cea-879e-437d-bc83-d508c22e7806","Type":"ContainerStarted","Data":"5563ecd635587e3c7c3a404bb0836e277456af282c41809e2cb190f6b683b9e0"}
Sep 19 12:36:18.422778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:18.422733    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2" podStartSLOduration=2.25735842 podStartE2EDuration="2.422717614s" totalImagesPullingTime="165.359194ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:36:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:36:18.421746088 +0000 UTC m=+661.781989029" watchObservedRunningTime="2026-09-19 12:36:18.422717614 +0000 UTC m=+661.782960554"
Sep 19 12:36:18.519622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:18.519579    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:36:26.026796 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.026761    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"]
Sep 19 12:36:26.031632 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.031611    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.034284 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.034261    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 19 12:36:26.039799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.039775    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"]
Sep 19 12:36:26.067000 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.066976    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:36:26.185916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.185869    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67ppd\" (UniqueName: \"kubernetes.io/projected/9b506a77-e70c-4392-8d44-9fb534659db0-kube-api-access-67ppd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.185916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.185933    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.186205 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.185963    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b506a77-e70c-4392-8d44-9fb534659db0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.186205 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.186008    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.186205 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.186104    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.186205 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.186139    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287104 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287015    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287104 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b506a77-e70c-4392-8d44-9fb534659db0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287104 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287074    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287104 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287103    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287117    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67ppd\" (UniqueName: \"kubernetes.io/projected/9b506a77-e70c-4392-8d44-9fb534659db0-kube-api-access-67ppd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287415    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287493    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.287649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.287571    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.289358 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.289337    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b506a77-e70c-4392-8d44-9fb534659db0-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.289693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.289674    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b506a77-e70c-4392-8d44-9fb534659db0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.294791 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.294774    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67ppd\" (UniqueName: \"kubernetes.io/projected/9b506a77-e70c-4392-8d44-9fb534659db0-kube-api-access-67ppd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-th7hj\" (UID: \"9b506a77-e70c-4392-8d44-9fb534659db0\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.341853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.341815    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:26.615184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:26.612776    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"]
Sep 19 12:36:27.037490 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.037445    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:27.037852 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.037513    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:27.049786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.049759    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:27.446773 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.446693    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj" event={"ID":"9b506a77-e70c-4392-8d44-9fb534659db0","Type":"ContainerStarted","Data":"c974a3832aa312f108e98dc339a94c0b00ae05c33bf44cde7005d6c5b27371ce"}
Sep 19 12:36:27.446773 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.446734    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj" event={"ID":"9b506a77-e70c-4392-8d44-9fb534659db0","Type":"ContainerStarted","Data":"9a26b5439d1c4d058e2070dc57af0237a9b7729873b222c707c0f6cbcb4d6f13"}
Sep 19 12:36:27.458012 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.457987    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-l6hl2"
Sep 19 12:36:27.463549 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:27.463511    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj" podStartSLOduration=1.302050617 podStartE2EDuration="1.463497732s" totalImagesPullingTime="161.447115ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:36:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:36:27.461940433 +0000 UTC m=+670.822183375" watchObservedRunningTime="2026-09-19 12:36:27.463497732 +0000 UTC m=+670.823740674"
Sep 19 12:36:28.328013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:28.327975    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:36:36.342217 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:36.342154    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:36.342759 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:36.342311    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:36.354766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:36.354743    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:36:36.495901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:36:36.495865    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-th7hj"
Sep 19 12:37:40.828170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.828089    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-98dc84ff9-s62vg"]
Sep 19 12:37:40.831223 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.831200    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:40.834750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.834733    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 19 12:37:40.834812 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.834756    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ll9kr\""
Sep 19 12:37:40.839323 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.839301    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-98dc84ff9-s62vg"]
Sep 19 12:37:40.884634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.884607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/799bbd28-7132-4f43-acea-d8c6d48fd9f9-tls-cert\") pod \"authorino-98dc84ff9-s62vg\" (UID: \"799bbd28-7132-4f43-acea-d8c6d48fd9f9\") " pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:40.884634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.884639    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vrpg\" (UniqueName: \"kubernetes.io/projected/799bbd28-7132-4f43-acea-d8c6d48fd9f9-kube-api-access-9vrpg\") pod \"authorino-98dc84ff9-s62vg\" (UID: \"799bbd28-7132-4f43-acea-d8c6d48fd9f9\") " pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:40.985423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.985389    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/799bbd28-7132-4f43-acea-d8c6d48fd9f9-tls-cert\") pod \"authorino-98dc84ff9-s62vg\" (UID: \"799bbd28-7132-4f43-acea-d8c6d48fd9f9\") " pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:40.985600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.985428    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vrpg\" (UniqueName: \"kubernetes.io/projected/799bbd28-7132-4f43-acea-d8c6d48fd9f9-kube-api-access-9vrpg\") pod \"authorino-98dc84ff9-s62vg\" (UID: \"799bbd28-7132-4f43-acea-d8c6d48fd9f9\") " pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:40.987960 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.987939    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/799bbd28-7132-4f43-acea-d8c6d48fd9f9-tls-cert\") pod \"authorino-98dc84ff9-s62vg\" (UID: \"799bbd28-7132-4f43-acea-d8c6d48fd9f9\") " pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:40.992806 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:40.992783    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vrpg\" (UniqueName: \"kubernetes.io/projected/799bbd28-7132-4f43-acea-d8c6d48fd9f9-kube-api-access-9vrpg\") pod \"authorino-98dc84ff9-s62vg\" (UID: \"799bbd28-7132-4f43-acea-d8c6d48fd9f9\") " pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:41.141133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:41.141051    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-98dc84ff9-s62vg"
Sep 19 12:37:41.407855 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:41.407830    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-98dc84ff9-s62vg"]
Sep 19 12:37:41.409386 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:37:41.409359    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod799bbd28_7132_4f43_acea_d8c6d48fd9f9.slice/crio-c8157c2d22815b4e95050219d5b3610cca80ce77d9a3545c53e5cc762bce34a4 WatchSource:0}: Error finding container c8157c2d22815b4e95050219d5b3610cca80ce77d9a3545c53e5cc762bce34a4: Status 404 returned error can't find the container with id c8157c2d22815b4e95050219d5b3610cca80ce77d9a3545c53e5cc762bce34a4
Sep 19 12:37:41.762028 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:41.761993    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-98dc84ff9-s62vg" event={"ID":"799bbd28-7132-4f43-acea-d8c6d48fd9f9","Type":"ContainerStarted","Data":"c8157c2d22815b4e95050219d5b3610cca80ce77d9a3545c53e5cc762bce34a4"}
Sep 19 12:37:42.768316 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:42.768275    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-98dc84ff9-s62vg" event={"ID":"799bbd28-7132-4f43-acea-d8c6d48fd9f9","Type":"ContainerStarted","Data":"b2faf3c8cba24bcd2991ec9b111c72552ca51d6b404540f146600b6df4cd18bb"}
Sep 19 12:37:42.784281 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:42.784232    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-98dc84ff9-s62vg" podStartSLOduration=2.173941428 podStartE2EDuration="2.784216364s" totalImagesPullingTime="610.274936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:37:42.783069028 +0000 UTC m=+746.143311969" watchObservedRunningTime="2026-09-19 12:37:42.784216364 +0000 UTC m=+746.144459308"
Sep 19 12:37:43.301191 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:43.301149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:37:43.404518 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:43.404481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:37:44.176204 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:44.176166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:37:44.276683 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:44.276636    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:37:44.390623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:44.390581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:37:44.493630 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:37:44.493588    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:00.768035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:00.767989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:00.881878 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:00.881849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:00.984547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:00.984511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:01.117111 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:01.117035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:01.211878 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:01.211849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:01.304891 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:01.304853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:01.424326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:01.424241    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:01.677423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:01.677322    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:01.751989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:01.751951    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:02.115273 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:02.115243    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:02.214797 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:02.214770    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:02.570546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:02.570518    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:03.824626 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:03.824589    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:03.916789 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:03.916756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:04.232953 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:04.232919    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:38:05.031763 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:05.031727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:05.105600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:05.105560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:05.279978 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:05.279944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:05.665690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:05.665661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:05.768907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:05.768877    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:06.954229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:06.954184    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:07.161384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:07.161340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:07.247732 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:07.247631    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:07.347550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:07.347506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:07.858988 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:07.858945    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:08.228591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:08.228551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:08.521956 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:08.521866    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:08.678147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:08.678107    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:08.782437 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:08.782341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:08.887145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:08.887115    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:09.266203 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.266159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:09.367564 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.367530    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:09.829890 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.829861    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"]
Sep 19 12:38:09.832974 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.832953    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:09.836138 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.836113    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 12:38:09.837569 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.837548    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-04a668-serving-cert\""
Sep 19 12:38:09.837679 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.837595    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:38:09.837679 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.837650    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sr888\""
Sep 19 12:38:09.837868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.837853    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 12:38:09.845002 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.844981    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"]
Sep 19 12:38:09.873647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.873614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:09.914393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.914354    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:09.914594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.914397    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:09.914594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.914552    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-metrics-certs\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:09.914701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.914597    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkhp9\" (UniqueName: \"kubernetes.io/projected/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-kube-api-access-qkhp9\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:09.963452 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:09.963423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:10.015501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.015446    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-metrics-certs\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.015501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.015507    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkhp9\" (UniqueName: \"kubernetes.io/projected/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-kube-api-access-qkhp9\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.015738 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.015557    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.015738 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.015581    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.016184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.016160    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.017871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.017847    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-metrics-certs\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.017960 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.017907    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-maas-api-tls\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.023990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.023968    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkhp9\" (UniqueName: \"kubernetes.io/projected/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-kube-api-access-qkhp9\") pod \"maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.029032 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.029004    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:10.145485 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.145389    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:10.197453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.197416    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:10.417826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.417797    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"]
Sep 19 12:38:10.420367 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:10.420343    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a1d15a_79d8_40b5_b449_9fa6b4a4ca2c.slice/crio-bea0d8572737302a30399cea0d36c06a511f83fe5139496d70474b3ded420e8c WatchSource:0}: Error finding container bea0d8572737302a30399cea0d36c06a511f83fe5139496d70474b3ded420e8c: Status 404 returned error can't find the container with id bea0d8572737302a30399cea0d36c06a511f83fe5139496d70474b3ded420e8c
Sep 19 12:38:10.489721 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.489659    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:10.586984 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.586949    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:10.879382 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.879345    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:10.895496 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:10.895440    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94" event={"ID":"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c","Type":"ContainerStarted","Data":"bea0d8572737302a30399cea0d36c06a511f83fe5139496d70474b3ded420e8c"}
Sep 19 12:38:11.456552 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:11.456510    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:11.678307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:11.678270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:12.099048 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:12.098970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:12.240509 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:12.240472    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:12.631183 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:12.631139    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:12.907934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:12.907837    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94" event={"ID":"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c","Type":"ContainerStarted","Data":"f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf"}
Sep 19 12:38:12.907934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:12.907904    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:12.927386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:12.927339    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94" podStartSLOduration=1.718785 podStartE2EDuration="3.927322035s" totalImagesPullingTime="2.208537035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:12.926144732 +0000 UTC m=+776.286387684" watchObservedRunningTime="2026-09-19 12:38:12.927322035 +0000 UTC m=+776.287564972"
Sep 19 12:38:13.669387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:13.669355    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:13.767195 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:13.767154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:14.884333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:14.884298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:15.945533 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:15.945495    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:16.647931 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.647901    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"]
Sep 19 12:38:16.651494 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.651451    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.659491 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.654297    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-5f80f73c-serving-cert\""
Sep 19 12:38:16.666328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.666251    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"]
Sep 19 12:38:16.687482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.687442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:16.739932 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.739900    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:16.778842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.778816    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxkzs\" (UniqueName: \"kubernetes.io/projected/a2d1de8c-51f8-4509-a193-ce1f62006cc2-kube-api-access-rxkzs\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.778985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.778850    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-maas-api-tls\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.778985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.778884    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d1de8c-51f8-4509-a193-ce1f62006cc2-openshift-service-ca\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.778985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.778960    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-metrics-certs\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.840915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.840874    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:16.880068 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.880040    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxkzs\" (UniqueName: \"kubernetes.io/projected/a2d1de8c-51f8-4509-a193-ce1f62006cc2-kube-api-access-rxkzs\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.880206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.880071    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-maas-api-tls\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.880206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.880101    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d1de8c-51f8-4509-a193-ce1f62006cc2-openshift-service-ca\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.880311 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.880229    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-metrics-certs\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.880694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.880676    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d1de8c-51f8-4509-a193-ce1f62006cc2-openshift-service-ca\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.882678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.882660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-maas-api-tls\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.882734 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.882707    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-metrics-certs\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.887626 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.887607    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxkzs\" (UniqueName: \"kubernetes.io/projected/a2d1de8c-51f8-4509-a193-ce1f62006cc2-kube-api-access-rxkzs\") pod \"maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:16.972745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:16.972715    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:17.029347 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.029254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:17.196604 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.196574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:17.250328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.250304    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"]
Sep 19 12:38:17.252142 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:17.252103    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2d1de8c_51f8_4509_a193_ce1f62006cc2.slice/crio-d97995f10f710d1d347a76d554407895c2cf0be73e89bb8e634ad8c4c5bec05b WatchSource:0}: Error finding container d97995f10f710d1d347a76d554407895c2cf0be73e89bb8e634ad8c4c5bec05b: Status 404 returned error can't find the container with id d97995f10f710d1d347a76d554407895c2cf0be73e89bb8e634ad8c4c5bec05b
Sep 19 12:38:17.297152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.297122    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:17.408076 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.408042    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"]
Sep 19 12:38:17.412423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.412403    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.415314 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.415295    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5f80f73c-dockercfg-gl94d\""
Sep 19 12:38:17.415501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.415455    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5f80f73c\""
Sep 19 12:38:17.420430 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.420286    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"]
Sep 19 12:38:17.485740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.485709    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rj5ww\" (UniqueName: \"kubernetes.io/projected/b394dc31-a13c-4ba5-ad0e-248b309c6df2-kube-api-access-rj5ww\") pod \"payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.485908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.485761    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b394dc31-a13c-4ba5-ad0e-248b309c6df2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.586498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.586394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rj5ww\" (UniqueName: \"kubernetes.io/projected/b394dc31-a13c-4ba5-ad0e-248b309c6df2-kube-api-access-rj5ww\") pod \"payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.586498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.586442    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b394dc31-a13c-4ba5-ad0e-248b309c6df2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.587178 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.587160    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b394dc31-a13c-4ba5-ad0e-248b309c6df2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.594834 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.594816    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rj5ww\" (UniqueName: \"kubernetes.io/projected/b394dc31-a13c-4ba5-ad0e-248b309c6df2-kube-api-access-rj5ww\") pod \"payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.676651 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.676618    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:17.723995 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.723969    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:17.729124 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.729100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:17.935425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.935334    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr" event={"ID":"a2d1de8c-51f8-4509-a193-ce1f62006cc2","Type":"ContainerStarted","Data":"c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61"}
Sep 19 12:38:17.935425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.935389    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr" event={"ID":"a2d1de8c-51f8-4509-a193-ce1f62006cc2","Type":"ContainerStarted","Data":"d97995f10f710d1d347a76d554407895c2cf0be73e89bb8e634ad8c4c5bec05b"}
Sep 19 12:38:17.935425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.935413    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:17.953543 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.953489    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr" podStartSLOduration=1.519959641 podStartE2EDuration="1.95344857s" totalImagesPullingTime="433.488929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:17.952875668 +0000 UTC m=+781.313118610" watchObservedRunningTime="2026-09-19 12:38:17.95344857 +0000 UTC m=+781.313691515"
Sep 19 12:38:17.966774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:17.966742    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:18.024346 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.024325    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"]
Sep 19 12:38:18.025939 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:18.025915    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb394dc31_a13c_4ba5_ad0e_248b309c6df2.slice/crio-c7df09234cc62cd6ad929ad3327cf8f16263ac297a820b91b9e5e072e2175292 WatchSource:0}: Error finding container c7df09234cc62cd6ad929ad3327cf8f16263ac297a820b91b9e5e072e2175292: Status 404 returned error can't find the container with id c7df09234cc62cd6ad929ad3327cf8f16263ac297a820b91b9e5e072e2175292
Sep 19 12:38:18.300200 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.300167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:18.399654 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.399619    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:18.482453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.482417    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"]
Sep 19 12:38:18.485919 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.485902    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.488740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.488719    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-38aa0a-dockercfg-4tftm\""
Sep 19 12:38:18.488865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.488759    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-38aa0a\""
Sep 19 12:38:18.493634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.493615    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"]
Sep 19 12:38:18.512176 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.512141    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:18.594889 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.594817    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58967\" (UniqueName: \"kubernetes.io/projected/17589e31-e96b-4df8-a805-d4a17a6bbf56-kube-api-access-58967\") pod \"payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.595033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.594923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17589e31-e96b-4df8-a805-d4a17a6bbf56-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.695837 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.695803    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58967\" (UniqueName: \"kubernetes.io/projected/17589e31-e96b-4df8-a805-d4a17a6bbf56-kube-api-access-58967\") pod \"payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.696027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.695926    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17589e31-e96b-4df8-a805-d4a17a6bbf56-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.698535 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.698511    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-38aa0a\""
Sep 19 12:38:18.706995 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.706967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58967\" (UniqueName: \"kubernetes.io/projected/17589e31-e96b-4df8-a805-d4a17a6bbf56-kube-api-access-58967\") pod \"payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.707241 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.707217    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17589e31-e96b-4df8-a805-d4a17a6bbf56-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.800030 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.800002    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-38aa0a-dockercfg-4tftm\""
Sep 19 12:38:18.808310 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.808287    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:18.817610 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.817574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:18.923500 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.923451    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:38:18.942403 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.942361    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld" event={"ID":"b394dc31-a13c-4ba5-ad0e-248b309c6df2","Type":"ContainerStarted","Data":"c7df09234cc62cd6ad929ad3327cf8f16263ac297a820b91b9e5e072e2175292"}
Sep 19 12:38:18.976678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:18.976639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:19.102079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.102004    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"]
Sep 19 12:38:19.165528 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.165404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:19.272892 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:19.272860    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17589e31_e96b_4df8_a805_d4a17a6bbf56.slice/crio-3b76c88b734f7985d5a764974a19497320ee35cbbe60373206b4b10495d66f54 WatchSource:0}: Error finding container 3b76c88b734f7985d5a764974a19497320ee35cbbe60373206b4b10495d66f54: Status 404 returned error can't find the container with id 3b76c88b734f7985d5a764974a19497320ee35cbbe60373206b4b10495d66f54
Sep 19 12:38:19.310323 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.310292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:19.777523 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.777482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:19.877695 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.877660    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:19.950585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.950479    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n" event={"ID":"17589e31-e96b-4df8-a805-d4a17a6bbf56","Type":"ContainerStarted","Data":"3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10"}
Sep 19 12:38:19.950585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.950542    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n" event={"ID":"17589e31-e96b-4df8-a805-d4a17a6bbf56","Type":"ContainerStarted","Data":"3b76c88b734f7985d5a764974a19497320ee35cbbe60373206b4b10495d66f54"}
Sep 19 12:38:19.950811 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.950734    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:19.952222 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.952199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld" event={"ID":"b394dc31-a13c-4ba5-ad0e-248b309c6df2","Type":"ContainerStarted","Data":"14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253"}
Sep 19 12:38:19.952355 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.952332    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:19.971192 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.971141    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n" podStartSLOduration=1.5618623280000001 podStartE2EDuration="1.9711253s" totalImagesPullingTime="409.262972ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:19.969786441 +0000 UTC m=+783.330029384" watchObservedRunningTime="2026-09-19 12:38:19.9711253 +0000 UTC m=+783.331368238"
Sep 19 12:38:19.975497 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.975443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:19.986869 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:19.986829    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld" podStartSLOduration=1.696432471 podStartE2EDuration="2.986815496s" totalImagesPullingTime="1.290383025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:19.985173837 +0000 UTC m=+783.345416781" watchObservedRunningTime="2026-09-19 12:38:19.986815496 +0000 UTC m=+783.347058437"
Sep 19 12:38:20.043520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:20.043485    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:20.138986 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:20.138951    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:20.591572 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:20.591534    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:21.466306 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:21.466271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:22.115641 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:22.115609    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:22.202232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:22.202189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:22.517754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:22.517721    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:22.822936 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:22.822824    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:23.530252 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.530220    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"]
Sep 19 12:38:23.534003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.533976    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.536972 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.536946    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5be349-serving-cert\""
Sep 19 12:38:23.544797 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.544773    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"]
Sep 19 12:38:23.569069 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.569037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:23.633408 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.633352    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:23.643451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.643424    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-maas-api-tls\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.643451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.643456    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9nf6\" (UniqueName: \"kubernetes.io/projected/951394ab-552a-4fc9-a144-a6f4b0606896-kube-api-access-m9nf6\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.643636 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.643490    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951394ab-552a-4fc9-a144-a6f4b0606896-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.643636 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.643608    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-metrics-certs\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.733645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.733612    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:23.744636 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.744613    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-maas-api-tls\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.744713 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.744643    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9nf6\" (UniqueName: \"kubernetes.io/projected/951394ab-552a-4fc9-a144-a6f4b0606896-kube-api-access-m9nf6\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.744713 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.744665    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951394ab-552a-4fc9-a144-a6f4b0606896-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.744792 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.744731    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-metrics-certs\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.745390 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.745370    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951394ab-552a-4fc9-a144-a6f4b0606896-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.747127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.747102    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-metrics-certs\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.747127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.747110    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-maas-api-tls\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.752631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.752609    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9nf6\" (UniqueName: \"kubernetes.io/projected/951394ab-552a-4fc9-a144-a6f4b0606896-kube-api-access-m9nf6\") pod \"maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.846321 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.846264    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:23.948168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:23.948145    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:24.070826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.070788    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:24.120673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.120650    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"]
Sep 19 12:38:24.122432 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:24.122407    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod951394ab_552a_4fc9_a144_a6f4b0606896.slice/crio-45d40b36d455ff2da57d1279ea4432cce03995d493d4e529586703f755e929a4 WatchSource:0}: Error finding container 45d40b36d455ff2da57d1279ea4432cce03995d493d4e529586703f755e929a4: Status 404 returned error can't find the container with id 45d40b36d455ff2da57d1279ea4432cce03995d493d4e529586703f755e929a4
Sep 19 12:38:24.162753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.162708    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:24.473053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.473025    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:24.945364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.945290    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:38:24.977656 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.977622    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r" event={"ID":"951394ab-552a-4fc9-a144-a6f4b0606896","Type":"ContainerStarted","Data":"87b5aaaee728ba70d589a0982457cda13a50e3259681bfb895c09d0fe45f2a4e"}
Sep 19 12:38:24.977808 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.977678    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r" event={"ID":"951394ab-552a-4fc9-a144-a6f4b0606896","Type":"ContainerStarted","Data":"45d40b36d455ff2da57d1279ea4432cce03995d493d4e529586703f755e929a4"}
Sep 19 12:38:24.977808 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.977705    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:24.996644 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:24.996601    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r" podStartSLOduration=1.6563720069999999 podStartE2EDuration="1.996586983s" totalImagesPullingTime="340.214976ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:24.994758868 +0000 UTC m=+788.355001810" watchObservedRunningTime="2026-09-19 12:38:24.996586983 +0000 UTC m=+788.356829952"
Sep 19 12:38:25.095707 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:25.095667    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:25.492145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:25.492099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:26.265660 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:26.265624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:26.746005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:26.745970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:27.164998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:27.164922    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:27.262356 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:27.262326    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:27.360717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:27.360682    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:28.502752 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:28.502714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:29.883735 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:29.883688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:30.913340 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:30.913302    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:30.958626 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:30.958599    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:30.958781 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:30.958667    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:38:30.988130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:30.988101    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:38:31.759707 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:31.759672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:31.994567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:31.994528    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:32.138393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:32.138300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:32.423264 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:32.423163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:32.522184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:32.522149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:32.620922 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:32.620880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:33.177446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:33.177407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:34.036332 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:34.036296    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:34.483349 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:34.483304    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:34.579238 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:34.579201    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:34.730114 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:34.730039    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:34.947189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:34.947102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:35.703901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:35.703856    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:36.497950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:36.497917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:36.811813 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:36.811732    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:37.257955 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:37.257928    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:38.082866 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:38.082831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:38.904925 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:38.904880    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:39.893935 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:39.893889    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:40.666101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:40.666062    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:40.787208 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:40.787172    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:40.883696 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:40.883662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:41.011369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:41.011310    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:42.607067 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:42.607030    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:43.851992 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:43.851957    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:44.561696 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.561667    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"]
Sep 19 12:38:44.561962 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.561937    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr" podUID="a2d1de8c-51f8-4509-a193-ce1f62006cc2" containerName="maas-api" containerID="cri-o://c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61" gracePeriod=30
Sep 19 12:38:44.590970 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.590944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:44.661004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.660960    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"]
Sep 19 12:38:44.661256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.661227    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld" podUID="b394dc31-a13c-4ba5-ad0e-248b309c6df2" containerName="payload-pre-processing" containerID="cri-o://14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253" gracePeriod=30
Sep 19 12:38:44.699012 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.698970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:44.800383 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.800310    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:44.823734 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.823674    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:44.871449 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.871404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:44.898000 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.897979    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:44.922149 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922123    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxkzs\" (UniqueName: \"kubernetes.io/projected/a2d1de8c-51f8-4509-a193-ce1f62006cc2-kube-api-access-rxkzs\") pod \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") "
Sep 19 12:38:44.922321 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922172    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d1de8c-51f8-4509-a193-ce1f62006cc2-openshift-service-ca\") pod \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") "
Sep 19 12:38:44.922321 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922205    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-maas-api-tls\") pod \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") "
Sep 19 12:38:44.922321 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922250    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rj5ww\" (UniqueName: \"kubernetes.io/projected/b394dc31-a13c-4ba5-ad0e-248b309c6df2-kube-api-access-rj5ww\") pod \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") "
Sep 19 12:38:44.922511 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922375    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-metrics-certs\") pod \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\" (UID: \"a2d1de8c-51f8-4509-a193-ce1f62006cc2\") "
Sep 19 12:38:44.922511 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922451    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b394dc31-a13c-4ba5-ad0e-248b309c6df2-plugins-config-volume\") pod \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\" (UID: \"b394dc31-a13c-4ba5-ad0e-248b309c6df2\") "
Sep 19 12:38:44.922661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922632    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2d1de8c-51f8-4509-a193-ce1f62006cc2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a2d1de8c-51f8-4509-a193-ce1f62006cc2" (UID: "a2d1de8c-51f8-4509-a193-ce1f62006cc2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:38:44.922766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.922749    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2d1de8c-51f8-4509-a193-ce1f62006cc2-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:38:44.923068 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.923039    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b394dc31-a13c-4ba5-ad0e-248b309c6df2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b394dc31-a13c-4ba5-ad0e-248b309c6df2" (UID: "b394dc31-a13c-4ba5-ad0e-248b309c6df2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:38:44.925130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.925102    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a2d1de8c-51f8-4509-a193-ce1f62006cc2" (UID: "a2d1de8c-51f8-4509-a193-ce1f62006cc2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:38:44.925236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.925147    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2d1de8c-51f8-4509-a193-ce1f62006cc2-kube-api-access-rxkzs" (OuterVolumeSpecName: "kube-api-access-rxkzs") pod "a2d1de8c-51f8-4509-a193-ce1f62006cc2" (UID: "a2d1de8c-51f8-4509-a193-ce1f62006cc2"). InnerVolumeSpecName "kube-api-access-rxkzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:38:44.925236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.925202    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b394dc31-a13c-4ba5-ad0e-248b309c6df2-kube-api-access-rj5ww" (OuterVolumeSpecName: "kube-api-access-rj5ww") pod "b394dc31-a13c-4ba5-ad0e-248b309c6df2" (UID: "b394dc31-a13c-4ba5-ad0e-248b309c6df2"). InnerVolumeSpecName "kube-api-access-rj5ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:38:44.925352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.925318    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a2d1de8c-51f8-4509-a193-ce1f62006cc2" (UID: "a2d1de8c-51f8-4509-a193-ce1f62006cc2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:38:44.995709 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:44.995677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:45.023138 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.023109    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxkzs\" (UniqueName: \"kubernetes.io/projected/a2d1de8c-51f8-4509-a193-ce1f62006cc2-kube-api-access-rxkzs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.023138 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.023132    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.023138 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.023142    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rj5ww\" (UniqueName: \"kubernetes.io/projected/b394dc31-a13c-4ba5-ad0e-248b309c6df2-kube-api-access-rj5ww\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.023345 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.023150    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a2d1de8c-51f8-4509-a193-ce1f62006cc2-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.023345 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.023159    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b394dc31-a13c-4ba5-ad0e-248b309c6df2-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:38:45.074815 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.074724    2571 generic.go:356] "Generic (PLEG): container finished" podID="a2d1de8c-51f8-4509-a193-ce1f62006cc2" containerID="c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61" exitCode=0
Sep 19 12:38:45.074815 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.074795    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"
Sep 19 12:38:45.075033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.074816    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr" event={"ID":"a2d1de8c-51f8-4509-a193-ce1f62006cc2","Type":"ContainerDied","Data":"c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61"}
Sep 19 12:38:45.075033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.074896    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr" event={"ID":"a2d1de8c-51f8-4509-a193-ce1f62006cc2","Type":"ContainerDied","Data":"d97995f10f710d1d347a76d554407895c2cf0be73e89bb8e634ad8c4c5bec05b"}
Sep 19 12:38:45.075033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.074938    2571 scope.go:117] "RemoveContainer" containerID="c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61"
Sep 19 12:38:45.076409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.076387    2571 generic.go:356] "Generic (PLEG): container finished" podID="b394dc31-a13c-4ba5-ad0e-248b309c6df2" containerID="14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253" exitCode=0
Sep 19 12:38:45.076539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.076435    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld" event={"ID":"b394dc31-a13c-4ba5-ad0e-248b309c6df2","Type":"ContainerDied","Data":"14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253"}
Sep 19 12:38:45.076539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.076484    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"
Sep 19 12:38:45.076539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.076490    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld" event={"ID":"b394dc31-a13c-4ba5-ad0e-248b309c6df2","Type":"ContainerDied","Data":"c7df09234cc62cd6ad929ad3327cf8f16263ac297a820b91b9e5e072e2175292"}
Sep 19 12:38:45.084008 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.083984    2571 scope.go:117] "RemoveContainer" containerID="c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61"
Sep 19 12:38:45.084275 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:38:45.084251    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61\": container with ID starting with c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61 not found: ID does not exist" containerID="c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61"
Sep 19 12:38:45.084369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.084286    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61"} err="failed to get container status \"c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61\": rpc error: code = NotFound desc = could not find container \"c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61\": container with ID starting with c2b3b39813ef381588352c4a6c645da150703b9e198f6a686a97f22f61c66f61 not found: ID does not exist"
Sep 19 12:38:45.084369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.084310    2571 scope.go:117] "RemoveContainer" containerID="14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253"
Sep 19 12:38:45.094480 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.094438    2571 scope.go:117] "RemoveContainer" containerID="14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253"
Sep 19 12:38:45.094926 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:38:45.094910    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253\": container with ID starting with 14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253 not found: ID does not exist" containerID="14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253"
Sep 19 12:38:45.094991 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.094936    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253"} err="failed to get container status \"14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253\": rpc error: code = NotFound desc = could not find container \"14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253\": container with ID starting with 14775a3dc9dbb3fab3ec655579f2fcbbc32b8cbd0d7e50b8967b4590fc989253 not found: ID does not exist"
Sep 19 12:38:45.100950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.100928    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"]
Sep 19 12:38:45.106100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.106075    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-5f80f73c-676f5b8667-6c7lr"]
Sep 19 12:38:45.116326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.116301    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"]
Sep 19 12:38:45.121919 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.121899    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-5f80f73c-694c657bb-crcld"]
Sep 19 12:38:45.406346 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.406266    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:45.499103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.499066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:45.692770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.692700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:45.928785 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.928754    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"]
Sep 19 12:38:45.929169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.929091    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2d1de8c-51f8-4509-a193-ce1f62006cc2" containerName="maas-api"
Sep 19 12:38:45.929169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.929104    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2d1de8c-51f8-4509-a193-ce1f62006cc2" containerName="maas-api"
Sep 19 12:38:45.929169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.929125    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b394dc31-a13c-4ba5-ad0e-248b309c6df2" containerName="payload-pre-processing"
Sep 19 12:38:45.929169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.929131    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="b394dc31-a13c-4ba5-ad0e-248b309c6df2" containerName="payload-pre-processing"
Sep 19 12:38:45.929305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.929182    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2d1de8c-51f8-4509-a193-ce1f62006cc2" containerName="maas-api"
Sep 19 12:38:45.929305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.929191    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="b394dc31-a13c-4ba5-ad0e-248b309c6df2" containerName="payload-pre-processing"
Sep 19 12:38:45.933524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.933506    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:45.936358 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.936334    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 12:38:45.936358 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.936348    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-9b376ee6-gw-openshift-default-dockercfg-ld6xb\""
Sep 19 12:38:45.943909 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.943854    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"]
Sep 19 12:38:45.955028 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:45.954981    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:46.031839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.031807    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-token\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.031858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-socket\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.031887    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-envoy\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.031953    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-credential-socket\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.032023    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-certs\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.032104    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95c5de6-9488-45d9-8706-6445bdb5c715-istiod-ca-cert\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.032129    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-data\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.032155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-podinfo\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.032268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.032182    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mskl4\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-kube-api-access-mskl4\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.133550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133515    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-certs\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.133756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95c5de6-9488-45d9-8706-6445bdb5c715-istiod-ca-cert\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.133756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133606    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-data\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.133756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133627    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-podinfo\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.133756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133651    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mskl4\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-kube-api-access-mskl4\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.133756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133707    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-token\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-socket\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133908    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-envoy\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133947    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-credential-socket\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-certs\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.133998    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-data\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134280 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.134178    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-socket\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134280 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.134222    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-credential-socket\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.134674 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.134656    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95c5de6-9488-45d9-8706-6445bdb5c715-istiod-ca-cert\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.136163 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.136144    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-envoy\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.136384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.136218    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-podinfo\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.142328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.142306    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-token\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.142574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.142553    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mskl4\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-kube-api-access-mskl4\") pod \"e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") " pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.244274 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.244243    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:46.319304 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.319270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:46.450922 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.450886    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:38:46.526250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.526223    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"]
Sep 19 12:38:46.526898 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:46.526874    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc95c5de6_9488_45d9_8706_6445bdb5c715.slice/crio-5bf2ec114d025e328db71dd79d7f83f9a5c1dce591bb448eba84d48952aaf6e4 WatchSource:0}: Error finding container 5bf2ec114d025e328db71dd79d7f83f9a5c1dce591bb448eba84d48952aaf6e4: Status 404 returned error can't find the container with id 5bf2ec114d025e328db71dd79d7f83f9a5c1dce591bb448eba84d48952aaf6e4
Sep 19 12:38:46.645835 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.645803    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:46.740341 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.740302    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2d1de8c-51f8-4509-a193-ce1f62006cc2" path="/var/lib/kubelet/pods/a2d1de8c-51f8-4509-a193-ce1f62006cc2/volumes"
Sep 19 12:38:46.740742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:46.740721    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b394dc31-a13c-4ba5-ad0e-248b309c6df2" path="/var/lib/kubelet/pods/b394dc31-a13c-4ba5-ad0e-248b309c6df2/volumes"
Sep 19 12:38:47.048504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:47.048451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:47.090909 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:47.090872    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" event={"ID":"c95c5de6-9488-45d9-8706-6445bdb5c715","Type":"ContainerStarted","Data":"5bf2ec114d025e328db71dd79d7f83f9a5c1dce591bb448eba84d48952aaf6e4"}
Sep 19 12:38:47.457170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:47.457133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:47.558391 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:47.558359    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:47.930657 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:47.930574    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:48.649709 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:48.649637    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:48.964383 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:48.964342    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:48.964503 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:48.964419    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:48.964503 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:48.964449    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:38:49.103009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:49.102978    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" event={"ID":"c95c5de6-9488-45d9-8706-6445bdb5c715","Type":"ContainerStarted","Data":"9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"}
Sep 19 12:38:49.123854 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:49.123804    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podStartSLOduration=1.688714697 podStartE2EDuration="4.123789635s" totalImagesPullingTime="2.435074938s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:49.122408413 +0000 UTC m=+812.482651355" watchObservedRunningTime="2026-09-19 12:38:49.123789635 +0000 UTC m=+812.484032635"
Sep 19 12:38:49.245031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:49.244963    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:49.405937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:49.405901    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:50.246594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:50.246554    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:50.250313 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:50.250290    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:50.611023 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:50.610949    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:50.707144 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:50.707114    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:51.112417 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:51.112391    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:51.113354 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:51.113334    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:38:51.192101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:51.192070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:52.001759 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:52.001725    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:52.891241 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:52.891209    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:53.676218 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:53.676181    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:54.463349 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:54.463311    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:55.278044 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.278012    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"]
Sep 19 12:38:55.281574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.281557    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.284292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.284273    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-9b376ee6-serving-cert\""
Sep 19 12:38:55.294842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.294819    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"]
Sep 19 12:38:55.351906 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.351876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:55.415623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.415561    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-metrics-certs\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.415623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.415591    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.415776 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.415622    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6smq\" (UniqueName: \"kubernetes.io/projected/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-kube-api-access-s6smq\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.415776 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.415740    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.452455 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.452423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:55.516597 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.516575    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.516701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.516611    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-metrics-certs\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.516701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.516629    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.516701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.516655    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6smq\" (UniqueName: \"kubernetes.io/projected/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-kube-api-access-s6smq\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.516866 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:38:55.516749    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-serving-cert: secret "maas-api-e2e-migrate-9b376ee6-serving-cert" not found
Sep 19 12:38:55.516866 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:38:55.516815    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls podName:9f798e7a-ee6f-4b01-8768-b2be5f2c39fe nodeName:}" failed. No retries permitted until 2026-09-19 12:38:56.016801002 +0000 UTC m=+819.377043926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls") pod "maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" (UID: "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe") : secret "maas-api-e2e-migrate-9b376ee6-serving-cert" not found
Sep 19 12:38:55.517258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.517238    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-openshift-service-ca\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.518962 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.518944    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-metrics-certs\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.525911 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.525885    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6smq\" (UniqueName: \"kubernetes.io/projected/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-kube-api-access-s6smq\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:55.778900 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.778870    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:55.880758 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.880720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:55.958415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.958381    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:38:55.962389 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.962363    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"]
Sep 19 12:38:55.965976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.965957    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:55.968355 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.968338    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-9b376ee6\""
Sep 19 12:38:55.976023 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:55.975999    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"]
Sep 19 12:38:56.020673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.020649    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.020774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.020695    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvr6\" (UniqueName: \"kubernetes.io/projected/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-kube-api-access-7lvr6\") pod \"payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.020774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.020745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:56.023107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.023080    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls\") pod \"maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:56.121184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.121116    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvr6\" (UniqueName: \"kubernetes.io/projected/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-kube-api-access-7lvr6\") pod \"payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.121304 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.121209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.121867 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.121850    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.129761 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.129741    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvr6\" (UniqueName: \"kubernetes.io/projected/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-kube-api-access-7lvr6\") pod \"payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.191348 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.191325    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:56.276769 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.276742    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:56.459291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.459265    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"]
Sep 19 12:38:56.460387 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:56.460360    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f798e7a_ee6f_4b01_8768_b2be5f2c39fe.slice/crio-cf3070c6e298cc882ff44b34c8bca7d2028cb2d301265bfd739f33f99ca6caca WatchSource:0}: Error finding container cf3070c6e298cc882ff44b34c8bca7d2028cb2d301265bfd739f33f99ca6caca: Status 404 returned error can't find the container with id cf3070c6e298cc882ff44b34c8bca7d2028cb2d301265bfd739f33f99ca6caca
Sep 19 12:38:56.544747 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:56.544727    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"]
Sep 19 12:38:56.546422 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:38:56.546402    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3280ea66_321d_4ce1_9bce_1e2ea59bea1e.slice/crio-8c1ca5f1283f0472e89fd2bce8c926c94a4dbcf9b3f10a56917bc26882016dc3 WatchSource:0}: Error finding container 8c1ca5f1283f0472e89fd2bce8c926c94a4dbcf9b3f10a56917bc26882016dc3: Status 404 returned error can't find the container with id 8c1ca5f1283f0472e89fd2bce8c926c94a4dbcf9b3f10a56917bc26882016dc3
Sep 19 12:38:57.142575 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.142484    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" event={"ID":"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe","Type":"ContainerStarted","Data":"8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512"}
Sep 19 12:38:57.142575 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.142543    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" event={"ID":"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe","Type":"ContainerStarted","Data":"cf3070c6e298cc882ff44b34c8bca7d2028cb2d301265bfd739f33f99ca6caca"}
Sep 19 12:38:57.142784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.142674    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:38:57.144070 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.144049    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7" event={"ID":"3280ea66-321d-4ce1-9bce-1e2ea59bea1e","Type":"ContainerStarted","Data":"3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4"}
Sep 19 12:38:57.144157 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.144096    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:38:57.144157 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.144120    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7" event={"ID":"3280ea66-321d-4ce1-9bce-1e2ea59bea1e","Type":"ContainerStarted","Data":"8c1ca5f1283f0472e89fd2bce8c926c94a4dbcf9b3f10a56917bc26882016dc3"}
Sep 19 12:38:57.160562 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.160523    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" podStartSLOduration=1.865679986 podStartE2EDuration="2.160509885s" totalImagesPullingTime="294.829899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:57.158027708 +0000 UTC m=+820.518270649" watchObservedRunningTime="2026-09-19 12:38:57.160509885 +0000 UTC m=+820.520752830"
Sep 19 12:38:57.173664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:38:57.173628    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7" podStartSLOduration=2.173616161 podStartE2EDuration="2.173616161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:38:57.173165565 +0000 UTC m=+820.533408506" watchObservedRunningTime="2026-09-19 12:38:57.173616161 +0000 UTC m=+820.533859093"
Sep 19 12:39:02.739952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:02.739919    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:39:03.155136 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:03.155058    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:39:08.152315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:08.152274    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:39:12.223743 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:12.223703    2571 generic.go:356] "Generic (PLEG): container finished" podID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerID="9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995" exitCode=0
Sep 19 12:39:12.224166 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:12.223779    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" event={"ID":"c95c5de6-9488-45d9-8706-6445bdb5c715","Type":"ContainerDied","Data":"9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"}
Sep 19 12:39:12.224213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:12.224205    2571 scope.go:117] "RemoveContainer" containerID="9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"
Sep 19 12:39:12.226229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:12.226188    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:39:12.226337 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:12.226280    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:39:12.226337 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:12.226327    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:39:13.230305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:13.230270    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" event={"ID":"c95c5de6-9488-45d9-8706-6445bdb5c715","Type":"ContainerStarted","Data":"2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27"}
Sep 19 12:39:13.245232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:13.245205    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:39:13.245443 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:13.245411    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 19 12:39:13.245577 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:13.245507    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 19 12:39:14.244863 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:14.244821    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 19 12:39:14.245361 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:14.244889    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 19 12:39:15.245401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:15.245365    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 19 12:39:15.246094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:15.246065    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 19 12:39:16.125651 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.125619    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"]
Sep 19 12:39:16.131580 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.131561    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.136522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.134562    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"simulator-w0-a2e924-kserve-self-signed-certs\""
Sep 19 12:39:16.136522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.134790    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"kube-root-ca.crt\""
Sep 19 12:39:16.136522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.134976    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"default-dockercfg-7btsc\""
Sep 19 12:39:16.136522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.135152    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"openshift-service-ca.crt\""
Sep 19 12:39:16.140601 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.140544    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"]
Sep 19 12:39:16.162968 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.162936    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:16.245301 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.245271    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 19 12:39:16.245613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.245583    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 19 12:39:16.299402 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.299363    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tmp-dir\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.299600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.299428    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tls-certs\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.299600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.299549    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-dshm\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.299600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.299582    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-home\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.299726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.299608    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-model-cache\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.299726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.299626    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npzt6\" (UniqueName: \"kubernetes.io/projected/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-kube-api-access-npzt6\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400045    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tls-certs\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400108    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-dshm\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400128    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-home\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400156    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-model-cache\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400172    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npzt6\" (UniqueName: \"kubernetes.io/projected/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-kube-api-access-npzt6\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400202    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tmp-dir\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400597    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-home\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400611    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tmp-dir\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.400731 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.400675    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-model-cache\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.402435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.402417    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-dshm\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.402832 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.402812    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tls-certs\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.407596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.407576    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npzt6\" (UniqueName: \"kubernetes.io/projected/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-kube-api-access-npzt6\") pod \"simulator-w0-a2e924-kserve-7b8984fb68-fvtc6\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") " pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.458775 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.458735    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:16.729089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:16.729055    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"]
Sep 19 12:39:16.730150 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:39:16.730125    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51fe2b4a_6611_42dd_95a5_f4834e5a7caa.slice/crio-e14bb234ec4f0432a6caa9bd7e7816003dfff283a0c56d4b3d66a6b99f8dfc08 WatchSource:0}: Error finding container e14bb234ec4f0432a6caa9bd7e7816003dfff283a0c56d4b3d66a6b99f8dfc08: Status 404 returned error can't find the container with id e14bb234ec4f0432a6caa9bd7e7816003dfff283a0c56d4b3d66a6b99f8dfc08
Sep 19 12:39:17.245676 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:17.245631    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 19 12:39:17.246185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:17.245700    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 19 12:39:17.260873 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:17.260732    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6" event={"ID":"51fe2b4a-6611-42dd-95a5-f4834e5a7caa","Type":"ContainerStarted","Data":"ff7a1a3d235b0226c31305e2755ae8bf676b7ee7bcecdaafa9f68ab1f0db7301"}
Sep 19 12:39:17.260873 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:17.260783    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6" event={"ID":"51fe2b4a-6611-42dd-95a5-f4834e5a7caa","Type":"ContainerStarted","Data":"e14bb234ec4f0432a6caa9bd7e7816003dfff283a0c56d4b3d66a6b99f8dfc08"}
Sep 19 12:39:17.280447 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:17.280397    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6" podStartSLOduration=1.280381533 podStartE2EDuration="1.280381533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:39:17.278413388 +0000 UTC m=+840.638656328" watchObservedRunningTime="2026-09-19 12:39:17.280381533 +0000 UTC m=+840.640624475"
Sep 19 12:39:18.066638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:18.066605    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:18.245440 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:18.245401    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 19 12:39:18.245782 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:18.245743    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 19 12:39:18.828344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:18.828301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:19.248927 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:19.248903    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:39:19.249371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:19.249188    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:39:19.249808 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:19.249790    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:39:19.485833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:19.485795    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:20.121600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:20.121564    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:20.936184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:20.936147    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:39:20.982440 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:20.982387    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:21.856735 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:21.856698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:22.507609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:22.507578    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:23.233152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:23.233120    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:23.898203 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:23.898164    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:24.547314 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:24.547271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:25.366165 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:25.366127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:26.047944 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:26.047913    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:26.459375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:26.459342    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:26.459774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:26.459391    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:26.472069 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:26.472040    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:27.320690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:27.320658    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:39:31.743740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:31.743710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:32.423573 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.423544    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"]
Sep 19 12:39:32.423802 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.423779    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" podUID="9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" containerName="maas-api" containerID="cri-o://8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512" gracePeriod=30
Sep 19 12:39:32.450836 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.450805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:32.484918 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.484891    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"]
Sep 19 12:39:32.485125 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.485103    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7" podUID="3280ea66-321d-4ce1-9bce-1e2ea59bea1e" containerName="payload-pre-processing" containerID="cri-o://3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4" gracePeriod=30
Sep 19 12:39:32.551725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.551686    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:32.646369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.646343    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:32.703801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.703765    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:32.757793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.757769    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:39:32.763965 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.763939    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:39:32.773881 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.773852    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"]
Sep 19 12:39:32.774108 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.774081    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" containerID="cri-o://2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27" gracePeriod=30
Sep 19 12:39:32.803372 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.803344    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:32.851851 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.851830    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-metrics-certs\") pod \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") "
Sep 19 12:39:32.851989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.851876    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-openshift-service-ca\") pod \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") "
Sep 19 12:39:32.851989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.851913    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7lvr6\" (UniqueName: \"kubernetes.io/projected/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-kube-api-access-7lvr6\") pod \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") "
Sep 19 12:39:32.851989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.851943    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls\") pod \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") "
Sep 19 12:39:32.851989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.851985    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-plugins-config-volume\") pod \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\" (UID: \"3280ea66-321d-4ce1-9bce-1e2ea59bea1e\") "
Sep 19 12:39:32.852200 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.852055    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6smq\" (UniqueName: \"kubernetes.io/projected/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-kube-api-access-s6smq\") pod \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\" (UID: \"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe\") "
Sep 19 12:39:32.852357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.852326    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" (UID: "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:39:32.852357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.852345    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3280ea66-321d-4ce1-9bce-1e2ea59bea1e" (UID: "3280ea66-321d-4ce1-9bce-1e2ea59bea1e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:39:32.854007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.853977    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" (UID: "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:39:32.854115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.854034    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" (UID: "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:39:32.854115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.854055    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-kube-api-access-7lvr6" (OuterVolumeSpecName: "kube-api-access-7lvr6") pod "3280ea66-321d-4ce1-9bce-1e2ea59bea1e" (UID: "3280ea66-321d-4ce1-9bce-1e2ea59bea1e"). InnerVolumeSpecName "kube-api-access-7lvr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:39:32.854115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.854077    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-kube-api-access-s6smq" (OuterVolumeSpecName: "kube-api-access-s6smq") pod "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" (UID: "9f798e7a-ee6f-4b01-8768-b2be5f2c39fe"). InnerVolumeSpecName "kube-api-access-s6smq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:39:32.953232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.953206    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:32.953232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.953231    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:32.953388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.953244    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6smq\" (UniqueName: \"kubernetes.io/projected/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-kube-api-access-s6smq\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:32.953388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.953256    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:32.953388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.953267    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:32.953388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:32.953279    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7lvr6\" (UniqueName: \"kubernetes.io/projected/3280ea66-321d-4ce1-9bce-1e2ea59bea1e-kube-api-access-7lvr6\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:33.068750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.068719    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:33.218645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.218580    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:33.341022 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.340987    2571 generic.go:356] "Generic (PLEG): container finished" podID="3280ea66-321d-4ce1-9bce-1e2ea59bea1e" containerID="3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4" exitCode=0
Sep 19 12:39:33.341189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.341036    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"
Sep 19 12:39:33.341189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.341035    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7" event={"ID":"3280ea66-321d-4ce1-9bce-1e2ea59bea1e","Type":"ContainerDied","Data":"3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4"}
Sep 19 12:39:33.341189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.341104    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7" event={"ID":"3280ea66-321d-4ce1-9bce-1e2ea59bea1e","Type":"ContainerDied","Data":"8c1ca5f1283f0472e89fd2bce8c926c94a4dbcf9b3f10a56917bc26882016dc3"}
Sep 19 12:39:33.341189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.341135    2571 scope.go:117] "RemoveContainer" containerID="3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4"
Sep 19 12:39:33.342575 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.342552    2571 generic.go:356] "Generic (PLEG): container finished" podID="9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" containerID="8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512" exitCode=0
Sep 19 12:39:33.342685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.342593    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"
Sep 19 12:39:33.342685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.342607    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" event={"ID":"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe","Type":"ContainerDied","Data":"8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512"}
Sep 19 12:39:33.342685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.342649    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2" event={"ID":"9f798e7a-ee6f-4b01-8768-b2be5f2c39fe","Type":"ContainerDied","Data":"cf3070c6e298cc882ff44b34c8bca7d2028cb2d301265bfd739f33f99ca6caca"}
Sep 19 12:39:33.349676 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.349656    2571 scope.go:117] "RemoveContainer" containerID="3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4"
Sep 19 12:39:33.349938 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:39:33.349921    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4\": container with ID starting with 3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4 not found: ID does not exist" containerID="3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4"
Sep 19 12:39:33.350003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.349945    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4"} err="failed to get container status \"3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4\": rpc error: code = NotFound desc = could not find container \"3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4\": container with ID starting with 3d4696fa13eceac273b258fdb41d562eaa76b624f76b71c19f1cc488edd2c8d4 not found: ID does not exist"
Sep 19 12:39:33.350003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.349961    2571 scope.go:117] "RemoveContainer" containerID="8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512"
Sep 19 12:39:33.357947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.357932    2571 scope.go:117] "RemoveContainer" containerID="8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512"
Sep 19 12:39:33.358204 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:39:33.358180    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512\": container with ID starting with 8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512 not found: ID does not exist" containerID="8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512"
Sep 19 12:39:33.358264 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.358210    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512"} err="failed to get container status \"8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512\": rpc error: code = NotFound desc = could not find container \"8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512\": container with ID starting with 8ae06463781ba031c6a82042bfbe1042f4278a9fc8ee6bf91233bd05a33c3512 not found: ID does not exist"
Sep 19 12:39:33.366896 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.366874    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"]
Sep 19 12:39:33.370281 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.370261    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-9b376ee6-f76f5b754-rkvj7"]
Sep 19 12:39:33.380121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.380103    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"]
Sep 19 12:39:33.384042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.384025    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-9b376ee6-9dfdcf747-9jps2"]
Sep 19 12:39:33.529576 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.529491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:33.785967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.785867    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:33.959740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:33.959710    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:34.013783 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.013754    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"]
Sep 19 12:39:34.014094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.014080    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3280ea66-321d-4ce1-9bce-1e2ea59bea1e" containerName="payload-pre-processing"
Sep 19 12:39:34.014094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.014094    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3280ea66-321d-4ce1-9bce-1e2ea59bea1e" containerName="payload-pre-processing"
Sep 19 12:39:34.014233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.014111    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" containerName="maas-api"
Sep 19 12:39:34.014233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.014117    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" containerName="maas-api"
Sep 19 12:39:34.014233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.014185    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" containerName="maas-api"
Sep 19 12:39:34.014233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.014206    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3280ea66-321d-4ce1-9bce-1e2ea59bea1e" containerName="payload-pre-processing"
Sep 19 12:39:34.016988 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.016973    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.019512 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.019494    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-38aa0a\"/\"default-dockercfg-7ghfn\""
Sep 19 12:39:34.019625 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.019493    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-38aa0a\"/\"openshift-service-ca.crt\""
Sep 19 12:39:34.020612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.020593    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-autores-38aa0a\"/\"autores-model-38aa0a-kserve-self-signed-certs\""
Sep 19 12:39:34.020694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.020613    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-autores-38aa0a\"/\"kube-root-ca.crt\""
Sep 19 12:39:34.025312 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.025290    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"]
Sep 19 12:39:34.063236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.063182    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77118fd8-2e1c-4734-840d-ec915f4fc149-tls-certs\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.063236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.063210    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8pvw\" (UniqueName: \"kubernetes.io/projected/77118fd8-2e1c-4734-840d-ec915f4fc149-kube-api-access-s8pvw\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.063355 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.063237    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-dshm\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.063355 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.063309    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-home\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.063424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.063365    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-model-cache\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.063424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.063384    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-tmp-dir\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.163986    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-model-cache\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164014    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-tmp-dir\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164030    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77118fd8-2e1c-4734-840d-ec915f4fc149-tls-certs\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8pvw\" (UniqueName: \"kubernetes.io/projected/77118fd8-2e1c-4734-840d-ec915f4fc149-kube-api-access-s8pvw\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-dshm\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164144    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-home\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164438 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164418    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-model-cache\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164509 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-home\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.164558 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.164539    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-tmp-dir\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.166297 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.166278    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-dshm\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.166698 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.166678    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77118fd8-2e1c-4734-840d-ec915f4fc149-tls-certs\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.174769 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.174752    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8pvw\" (UniqueName: \"kubernetes.io/projected/77118fd8-2e1c-4734-840d-ec915f4fc149-kube-api-access-s8pvw\") pod \"autores-model-38aa0a-kserve-6fcfcc446c-8nfcr\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") " pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.251513 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.251488    2571 patch_prober.go:28] interesting pod/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 19 12:39:34.251623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.251540    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 12:39:34.321329 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.321257    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:34.326186 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.326169    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:34.599222 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.599200    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"]
Sep 19 12:39:34.600917 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:39:34.600890    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77118fd8_2e1c_4734_840d_ec915f4fc149.slice/crio-7ee7f79eaeaaf4b184bf10a420cb73ac72a7920dfb0aa7a8b4feaff0102027b3 WatchSource:0}: Error finding container 7ee7f79eaeaaf4b184bf10a420cb73ac72a7920dfb0aa7a8b4feaff0102027b3: Status 404 returned error can't find the container with id 7ee7f79eaeaaf4b184bf10a420cb73ac72a7920dfb0aa7a8b4feaff0102027b3
Sep 19 12:39:34.648094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.648064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:34.744928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.744894    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3280ea66-321d-4ce1-9bce-1e2ea59bea1e" path="/var/lib/kubelet/pods/3280ea66-321d-4ce1-9bce-1e2ea59bea1e/volumes"
Sep 19 12:39:34.745365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:34.745345    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f798e7a-ee6f-4b01-8768-b2be5f2c39fe" path="/var/lib/kubelet/pods/9f798e7a-ee6f-4b01-8768-b2be5f2c39fe/volumes"
Sep 19 12:39:35.340231 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:35.340197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:35.357364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:35.357329    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr" event={"ID":"77118fd8-2e1c-4734-840d-ec915f4fc149","Type":"ContainerStarted","Data":"33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518"}
Sep 19 12:39:35.357516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:35.357372    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr" event={"ID":"77118fd8-2e1c-4734-840d-ec915f4fc149","Type":"ContainerStarted","Data":"7ee7f79eaeaaf4b184bf10a420cb73ac72a7920dfb0aa7a8b4feaff0102027b3"}
Sep 19 12:39:35.376797 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:35.376757    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr" podStartSLOduration=2.376743176 podStartE2EDuration="2.376743176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:39:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:39:35.374759729 +0000 UTC m=+858.735002671" watchObservedRunningTime="2026-09-19 12:39:35.376743176 +0000 UTC m=+858.736986155"
Sep 19 12:39:35.911913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:35.911874    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:36.012026 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:36.011997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:36.395996 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:36.395963    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:37.244368 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:37.244331    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:37.957005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:37.956976    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"]
Sep 19 12:39:37.960326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:37.960303    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:37.964809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:37.964749    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-450ffc30-serving-cert\""
Sep 19 12:39:37.976360 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:37.975690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"]
Sep 19 12:39:38.012141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.011071    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.012141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.011119    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/239bffbe-d513-4707-bba3-bd412ff8d3ad-openshift-service-ca\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.012141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.011156    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-metrics-certs\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.012141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.011240    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xw4rg\" (UniqueName: \"kubernetes.io/projected/239bffbe-d513-4707-bba3-bd412ff8d3ad-kube-api-access-xw4rg\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.012141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.012005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:38.044978 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.044922    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:39:38.073490 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.073443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:38.112098 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112076    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mskl4\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-kube-api-access-mskl4\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112207 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112120    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95c5de6-9488-45d9-8706-6445bdb5c715-istiod-ca-cert\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112207 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112183    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-socket\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112210    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-credential-socket\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112241    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-envoy\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112290    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-token\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112319    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-podinfo\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112554 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112353    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-data\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112554 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112389    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-certs\") pod \"c95c5de6-9488-45d9-8706-6445bdb5c715\" (UID: \"c95c5de6-9488-45d9-8706-6445bdb5c715\") "
Sep 19 12:39:38.112647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112557    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:39:38.112647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112568    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xw4rg\" (UniqueName: \"kubernetes.io/projected/239bffbe-d513-4707-bba3-bd412ff8d3ad-kube-api-access-xw4rg\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.112647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112633    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c95c5de6-9488-45d9-8706-6445bdb5c715-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:39:38.112803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112704    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.112803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112745    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/239bffbe-d513-4707-bba3-bd412ff8d3ad-openshift-service-ca\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.112914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112881    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-metrics-certs\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.112971 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112941    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c95c5de6-9488-45d9-8706-6445bdb5c715-istiod-ca-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.112971 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112961    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.113074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.112762    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:39:38.113074 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:39:38.112894    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-serving-cert: secret "maas-api-e2e-ait-450ffc30-serving-cert" not found
Sep 19 12:39:38.113182 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.113089    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-data" (OuterVolumeSpecName: "istio-data") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:39:38.113182 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.113038    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:39:38.113182 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:39:38.113118    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls podName:239bffbe-d513-4707-bba3-bd412ff8d3ad nodeName:}" failed. No retries permitted until 2026-09-19 12:39:38.613099986 +0000 UTC m=+861.973342912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls") pod "maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" (UID: "239bffbe-d513-4707-bba3-bd412ff8d3ad") : secret "maas-api-e2e-ait-450ffc30-serving-cert" not found
Sep 19 12:39:38.113696 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.113637    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/239bffbe-d513-4707-bba3-bd412ff8d3ad-openshift-service-ca\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.114840 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.114813    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-kube-api-access-mskl4" (OuterVolumeSpecName: "kube-api-access-mskl4") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "kube-api-access-mskl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:39:38.115116 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.115089    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-token" (OuterVolumeSpecName: "istio-token") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:39:38.115445 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.115425    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:39:38.115543 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.115442    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c95c5de6-9488-45d9-8706-6445bdb5c715" (UID: "c95c5de6-9488-45d9-8706-6445bdb5c715"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:39:38.115543 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.115490    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-metrics-certs\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.123233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.123213    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xw4rg\" (UniqueName: \"kubernetes.io/projected/239bffbe-d513-4707-bba3-bd412ff8d3ad-kube-api-access-xw4rg\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.214032 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214015    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-workload-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.214032 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214033    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mskl4\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-kube-api-access-mskl4\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.214148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214042    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-credential-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.214148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214051    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-envoy\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.214148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214059    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-token\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.214148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214066    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-podinfo\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.214148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.214074    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c95c5de6-9488-45d9-8706-6445bdb5c715-istio-data\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:39:38.318383 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.318274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:38.373606 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.373575    2571 generic.go:356] "Generic (PLEG): container finished" podID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerID="2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27" exitCode=0
Sep 19 12:39:38.373720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.373632    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"
Sep 19 12:39:38.373720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.373655    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" event={"ID":"c95c5de6-9488-45d9-8706-6445bdb5c715","Type":"ContainerDied","Data":"2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27"}
Sep 19 12:39:38.373720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.373708    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx" event={"ID":"c95c5de6-9488-45d9-8706-6445bdb5c715","Type":"ContainerDied","Data":"5bf2ec114d025e328db71dd79d7f83f9a5c1dce591bb448eba84d48952aaf6e4"}
Sep 19 12:39:38.373842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.373731    2571 scope.go:117] "RemoveContainer" containerID="2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27"
Sep 19 12:39:38.382354 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.382338    2571 scope.go:117] "RemoveContainer" containerID="9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"
Sep 19 12:39:38.389903 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.389884    2571 scope.go:117] "RemoveContainer" containerID="2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27"
Sep 19 12:39:38.390171 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:39:38.390153    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27\": container with ID starting with 2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27 not found: ID does not exist" containerID="2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27"
Sep 19 12:39:38.390219 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.390180    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27"} err="failed to get container status \"2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27\": rpc error: code = NotFound desc = could not find container \"2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27\": container with ID starting with 2b2109de810c924fc740a4fedca34d6e0fbcdc468465606ac05cdae40525cd27 not found: ID does not exist"
Sep 19 12:39:38.390219 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.390205    2571 scope.go:117] "RemoveContainer" containerID="9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"
Sep 19 12:39:38.390445 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:39:38.390429    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995\": container with ID starting with 9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995 not found: ID does not exist" containerID="9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"
Sep 19 12:39:38.390508 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.390451    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995"} err="failed to get container status \"9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995\": rpc error: code = NotFound desc = could not find container \"9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995\": container with ID starting with 9affcd83e8ed3feb58ebdb62d8936c12fbb48621d10e188fbbfe177518460995 not found: ID does not exist"
Sep 19 12:39:38.397939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.397918    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"]
Sep 19 12:39:38.402207 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.402181    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-9b376ee6-gw-openshift-default-5fcd8d687b-f46fx"]
Sep 19 12:39:38.517732 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.517700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:38.609679 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.609596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:38.617300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.617276    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.619531 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.619507    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls\") pod \"maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.648897 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.648875    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"]
Sep 19 12:39:38.649229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.649214    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy"
Sep 19 12:39:38.649273 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.649231    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy"
Sep 19 12:39:38.649273 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.649246    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy"
Sep 19 12:39:38.649273 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.649252    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy"
Sep 19 12:39:38.649379 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.649308    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy"
Sep 19 12:39:38.649379 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.649316    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" containerName="istio-proxy"
Sep 19 12:39:38.653617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.653603    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.656426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.656410    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-450ffc30\""
Sep 19 12:39:38.656531 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.656416    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-450ffc30-dockercfg-cm8fl\""
Sep 19 12:39:38.665619 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.665596    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"]
Sep 19 12:39:38.717717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.717691    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-plugins-config-volume\") pod \"payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") " pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.717834 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.717775    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp5rh\" (UniqueName: \"kubernetes.io/projected/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-kube-api-access-hp5rh\") pod \"payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") " pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.739715 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.739691    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c95c5de6-9488-45d9-8706-6445bdb5c715" path="/var/lib/kubelet/pods/c95c5de6-9488-45d9-8706-6445bdb5c715/volumes"
Sep 19 12:39:38.819109 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.819085    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp5rh\" (UniqueName: \"kubernetes.io/projected/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-kube-api-access-hp5rh\") pod \"payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") " pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.819236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.819119    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-plugins-config-volume\") pod \"payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") " pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.819745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.819729    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-plugins-config-volume\") pod \"payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") " pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.833359 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.833337    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp5rh\" (UniqueName: \"kubernetes.io/projected/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-kube-api-access-hp5rh\") pod \"payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") " pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.873794 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.873743    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:38.963492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.963454    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:38.997784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:38.997724    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:39.152513 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.152487    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"]
Sep 19 12:39:39.153041 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:39:39.153018    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239bffbe_d513_4707_bba3_bd412ff8d3ad.slice/crio-85b6b7f0465eaf86f6a0de432ab3f4f89457d3aa93947b2c249e234e635a42ea WatchSource:0}: Error finding container 85b6b7f0465eaf86f6a0de432ab3f4f89457d3aa93947b2c249e234e635a42ea: Status 404 returned error can't find the container with id 85b6b7f0465eaf86f6a0de432ab3f4f89457d3aa93947b2c249e234e635a42ea
Sep 19 12:39:39.181415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.181385    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:39.237279 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.237261    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"]
Sep 19 12:39:39.238317 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:39:39.238294    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c34e1c_be23_4cc5_b5c4_0745a8f983a4.slice/crio-06ec2f1b031a6f691a576f32f23b1647c255166aa95c6ad2f812d6a7ab1d371e WatchSource:0}: Error finding container 06ec2f1b031a6f691a576f32f23b1647c255166aa95c6ad2f812d6a7ab1d371e: Status 404 returned error can't find the container with id 06ec2f1b031a6f691a576f32f23b1647c255166aa95c6ad2f812d6a7ab1d371e
Sep 19 12:39:39.281413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.281385    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:39.337398 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.337368    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:39:39.382436 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.382395    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7" event={"ID":"78c34e1c-be23-4cc5-b5c4-0745a8f983a4","Type":"ContainerStarted","Data":"c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a"}
Sep 19 12:39:39.382637 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.382483    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7" event={"ID":"78c34e1c-be23-4cc5-b5c4-0745a8f983a4","Type":"ContainerStarted","Data":"06ec2f1b031a6f691a576f32f23b1647c255166aa95c6ad2f812d6a7ab1d371e"}
Sep 19 12:39:39.382637 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.382516    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:39.384967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.384944    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" event={"ID":"239bffbe-d513-4707-bba3-bd412ff8d3ad","Type":"ContainerStarted","Data":"85b6b7f0465eaf86f6a0de432ab3f4f89457d3aa93947b2c249e234e635a42ea"}
Sep 19 12:39:39.406951 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.406864    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7" podStartSLOduration=1.406845717 podStartE2EDuration="1.406845717s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:39:39.405885909 +0000 UTC m=+862.766128852" watchObservedRunningTime="2026-09-19 12:39:39.406845717 +0000 UTC m=+862.767088661"
Sep 19 12:39:39.987034 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:39.987002    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:40.392842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:40.392735    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" event={"ID":"239bffbe-d513-4707-bba3-bd412ff8d3ad","Type":"ContainerStarted","Data":"e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f"}
Sep 19 12:39:40.393006 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:40.392857    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:40.410762 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:40.410718    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" podStartSLOduration=2.929615349 podStartE2EDuration="3.410702391s" totalImagesPullingTime="481.087042ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:39:40.409436258 +0000 UTC m=+863.769679203" watchObservedRunningTime="2026-09-19 12:39:40.410702391 +0000 UTC m=+863.770945385"
Sep 19 12:39:40.765528 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:40.765498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:41.729878 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:41.729840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:39:44.327307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:44.327271    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:44.327307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:44.327314    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:44.339745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:44.339722    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:44.422546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:44.422518    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:39:46.402535 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:46.402501    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:39:50.394997 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:50.394967    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:39:57.223416 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:39:57.223376    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:40:01.013068 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:01.012989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:01.063191 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:01.063159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:03.211836 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.211794    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"]
Sep 19 12:40:03.216153 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.216136    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.220257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.220241    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"openshift-service-ca.crt\""
Sep 19 12:40:03.220365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.220259    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"kube-root-ca.crt\""
Sep 19 12:40:03.220365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.220244    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"default-dockercfg-lnbhr\""
Sep 19 12:40:03.220365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.220288    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"simulator-w4-04a668-kserve-self-signed-certs\""
Sep 19 12:40:03.222986 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.222911    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"]
Sep 19 12:40:03.313226 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.313196    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:03.340372 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.340341    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-tmp-dir\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.340540 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.340394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-model-cache\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.340540 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.340423    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-home\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.340540 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.340515    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9076e402-0fa9-4f76-882a-f7a927445602-tls-certs\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.340680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.340542    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-dshm\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.340680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.340566    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pvfw\" (UniqueName: \"kubernetes.io/projected/9076e402-0fa9-4f76-882a-f7a927445602-kube-api-access-2pvfw\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441174 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441148    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-model-cache\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-home\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441208    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9076e402-0fa9-4f76-882a-f7a927445602-tls-certs\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441224    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-dshm\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441300 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441247    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pvfw\" (UniqueName: \"kubernetes.io/projected/9076e402-0fa9-4f76-882a-f7a927445602-kube-api-access-2pvfw\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441583 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-tmp-dir\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441644 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441623    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-model-cache\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441692 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441676    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-home\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.441729 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.441696    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-tmp-dir\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.443470 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.443441    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-dshm\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.443856 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.443837    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9076e402-0fa9-4f76-882a-f7a927445602-tls-certs\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.449322 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.449294    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pvfw\" (UniqueName: \"kubernetes.io/projected/9076e402-0fa9-4f76-882a-f7a927445602-kube-api-access-2pvfw\") pod \"simulator-w4-04a668-kserve-85499c78b6-fhnpl\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") " pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.526189 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.526131    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:03.554888 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.554859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:03.795999 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:03.795976    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"]
Sep 19 12:40:03.797907 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:40:03.797880    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9076e402_0fa9_4f76_882a_f7a927445602.slice/crio-8763dbf191d2ede4c45c778f04d983fefc4149d6648c7f99e10bfaade5a619c7 WatchSource:0}: Error finding container 8763dbf191d2ede4c45c778f04d983fefc4149d6648c7f99e10bfaade5a619c7: Status 404 returned error can't find the container with id 8763dbf191d2ede4c45c778f04d983fefc4149d6648c7f99e10bfaade5a619c7
Sep 19 12:40:04.226996 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:04.226964    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:04.507826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:04.507743    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl" event={"ID":"9076e402-0fa9-4f76-882a-f7a927445602","Type":"ContainerStarted","Data":"ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b"}
Sep 19 12:40:04.507826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:04.507781    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl" event={"ID":"9076e402-0fa9-4f76-882a-f7a927445602","Type":"ContainerStarted","Data":"8763dbf191d2ede4c45c778f04d983fefc4149d6648c7f99e10bfaade5a619c7"}
Sep 19 12:40:04.525829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:04.525784    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl" podStartSLOduration=1.525766927 podStartE2EDuration="1.525766927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:04.524155783 +0000 UTC m=+887.884398724" watchObservedRunningTime="2026-09-19 12:40:04.525766927 +0000 UTC m=+887.886009867"
Sep 19 12:40:05.058065 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:05.058035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:05.793020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:05.792980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:06.441450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:06.441408    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:07.363779 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:07.363750    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:08.038519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:08.038484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:08.716719 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:08.716688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:09.379753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:09.379720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:09.530632 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:09.530589    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:40:10.361856 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:10.361826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:11.062087 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:11.062052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:11.708750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:11.708712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:12.471323 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:12.471291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:13.141072 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.141037    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:13.526255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.526220    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:13.526446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.526274    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:13.538450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.538430    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:13.561345 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.561316    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:40:13.823687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.823588    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:13.922520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:13.922485    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:14.664375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:14.664346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:17.428834 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:17.428803    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.077805 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.077773    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"]
Sep 19 12:40:18.078065 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.078038    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" podUID="239bffbe-d513-4707-bba3-bd412ff8d3ad" containerName="maas-api" containerID="cri-o://e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f" gracePeriod=30
Sep 19 12:40:18.102560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.102528    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.124965 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.124936    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"]
Sep 19 12:40:18.125210 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.125154    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7" podUID="78c34e1c-be23-4cc5-b5c4-0745a8f983a4" containerName="payload-processing" containerID="cri-o://c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a" gracePeriod=30
Sep 19 12:40:18.166857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.166829    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.230537 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.230448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.305342 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.305291    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.364427 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.364394    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.368726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.368707    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:40:18.393246 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.393220    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:40:18.471730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.471699    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls\") pod \"239bffbe-d513-4707-bba3-bd412ff8d3ad\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") "
Sep 19 12:40:18.471730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.471734    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/239bffbe-d513-4707-bba3-bd412ff8d3ad-openshift-service-ca\") pod \"239bffbe-d513-4707-bba3-bd412ff8d3ad\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") "
Sep 19 12:40:18.472240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.471786    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp5rh\" (UniqueName: \"kubernetes.io/projected/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-kube-api-access-hp5rh\") pod \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") "
Sep 19 12:40:18.472240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.471817    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-plugins-config-volume\") pod \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\" (UID: \"78c34e1c-be23-4cc5-b5c4-0745a8f983a4\") "
Sep 19 12:40:18.472240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.471900    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-metrics-certs\") pod \"239bffbe-d513-4707-bba3-bd412ff8d3ad\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") "
Sep 19 12:40:18.472240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.471928    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xw4rg\" (UniqueName: \"kubernetes.io/projected/239bffbe-d513-4707-bba3-bd412ff8d3ad-kube-api-access-xw4rg\") pod \"239bffbe-d513-4707-bba3-bd412ff8d3ad\" (UID: \"239bffbe-d513-4707-bba3-bd412ff8d3ad\") "
Sep 19 12:40:18.472240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.472168    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239bffbe-d513-4707-bba3-bd412ff8d3ad-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "239bffbe-d513-4707-bba3-bd412ff8d3ad" (UID: "239bffbe-d513-4707-bba3-bd412ff8d3ad"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:18.472550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.472236    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "78c34e1c-be23-4cc5-b5c4-0745a8f983a4" (UID: "78c34e1c-be23-4cc5-b5c4-0745a8f983a4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:18.474009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.473965    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "239bffbe-d513-4707-bba3-bd412ff8d3ad" (UID: "239bffbe-d513-4707-bba3-bd412ff8d3ad"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:40:18.474120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.474052    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "239bffbe-d513-4707-bba3-bd412ff8d3ad" (UID: "239bffbe-d513-4707-bba3-bd412ff8d3ad"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:40:18.474120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.474066    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239bffbe-d513-4707-bba3-bd412ff8d3ad-kube-api-access-xw4rg" (OuterVolumeSpecName: "kube-api-access-xw4rg") pod "239bffbe-d513-4707-bba3-bd412ff8d3ad" (UID: "239bffbe-d513-4707-bba3-bd412ff8d3ad"). InnerVolumeSpecName "kube-api-access-xw4rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:18.474120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.474093    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-kube-api-access-hp5rh" (OuterVolumeSpecName: "kube-api-access-hp5rh") pod "78c34e1c-be23-4cc5-b5c4-0745a8f983a4" (UID: "78c34e1c-be23-4cc5-b5c4-0745a8f983a4"). InnerVolumeSpecName "kube-api-access-hp5rh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:18.572899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.572867    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.572899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.572892    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/239bffbe-d513-4707-bba3-bd412ff8d3ad-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.572899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.572902    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp5rh\" (UniqueName: \"kubernetes.io/projected/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-kube-api-access-hp5rh\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.572899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.572911    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/78c34e1c-be23-4cc5-b5c4-0745a8f983a4-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.573160 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.572919    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/239bffbe-d513-4707-bba3-bd412ff8d3ad-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.573160 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.572927    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xw4rg\" (UniqueName: \"kubernetes.io/projected/239bffbe-d513-4707-bba3-bd412ff8d3ad-kube-api-access-xw4rg\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:18.574858 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.574830    2571 generic.go:356] "Generic (PLEG): container finished" podID="239bffbe-d513-4707-bba3-bd412ff8d3ad" containerID="e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f" exitCode=0
Sep 19 12:40:18.574983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.574879    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"
Sep 19 12:40:18.574983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.574909    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" event={"ID":"239bffbe-d513-4707-bba3-bd412ff8d3ad","Type":"ContainerDied","Data":"e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f"}
Sep 19 12:40:18.574983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.574951    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx" event={"ID":"239bffbe-d513-4707-bba3-bd412ff8d3ad","Type":"ContainerDied","Data":"85b6b7f0465eaf86f6a0de432ab3f4f89457d3aa93947b2c249e234e635a42ea"}
Sep 19 12:40:18.574983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.574978    2571 scope.go:117] "RemoveContainer" containerID="e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f"
Sep 19 12:40:18.576286 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.576265    2571 generic.go:356] "Generic (PLEG): container finished" podID="78c34e1c-be23-4cc5-b5c4-0745a8f983a4" containerID="c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a" exitCode=0
Sep 19 12:40:18.576386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.576300    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"
Sep 19 12:40:18.576386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.576332    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7" event={"ID":"78c34e1c-be23-4cc5-b5c4-0745a8f983a4","Type":"ContainerDied","Data":"c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a"}
Sep 19 12:40:18.576386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.576373    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7" event={"ID":"78c34e1c-be23-4cc5-b5c4-0745a8f983a4","Type":"ContainerDied","Data":"06ec2f1b031a6f691a576f32f23b1647c255166aa95c6ad2f812d6a7ab1d371e"}
Sep 19 12:40:18.585637 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.585618    2571 scope.go:117] "RemoveContainer" containerID="e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f"
Sep 19 12:40:18.585954 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:40:18.585933    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f\": container with ID starting with e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f not found: ID does not exist" containerID="e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f"
Sep 19 12:40:18.586020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.585966    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f"} err="failed to get container status \"e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f\": rpc error: code = NotFound desc = could not find container \"e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f\": container with ID starting with e1513a834a4283bdddefffb1226cdd5b29327a03e413977a8d00ac38b32ce39f not found: ID does not exist"
Sep 19 12:40:18.586020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.585990    2571 scope.go:117] "RemoveContainer" containerID="c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a"
Sep 19 12:40:18.593793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.593778    2571 scope.go:117] "RemoveContainer" containerID="c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a"
Sep 19 12:40:18.594033 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:40:18.594016    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a\": container with ID starting with c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a not found: ID does not exist" containerID="c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a"
Sep 19 12:40:18.594082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.594038    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a"} err="failed to get container status \"c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a\": rpc error: code = NotFound desc = could not find container \"c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a\": container with ID starting with c44ff827a72b1229a18212e3d9732b7b7200213d474b18e534c63c75ff73297a not found: ID does not exist"
Sep 19 12:40:18.598229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.598172    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"]
Sep 19 12:40:18.599890 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.599870    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-450ffc30-5c9f5c97f8-5vgxx"]
Sep 19 12:40:18.610007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.609988    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"]
Sep 19 12:40:18.614175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.614150    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-450ffc30-7d76b68f89-zlxh7"]
Sep 19 12:40:18.738125 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.738098    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:18.740883 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.740859    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="239bffbe-d513-4707-bba3-bd412ff8d3ad" path="/var/lib/kubelet/pods/239bffbe-d513-4707-bba3-bd412ff8d3ad/volumes"
Sep 19 12:40:18.741424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:18.741403    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78c34e1c-be23-4cc5-b5c4-0745a8f983a4" path="/var/lib/kubelet/pods/78c34e1c-be23-4cc5-b5c4-0745a8f983a4/volumes"
Sep 19 12:40:19.396906 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:19.396815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:20.080473 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080430    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"]
Sep 19 12:40:20.080851 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080834    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78c34e1c-be23-4cc5-b5c4-0745a8f983a4" containerName="payload-processing"
Sep 19 12:40:20.080892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080853    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="78c34e1c-be23-4cc5-b5c4-0745a8f983a4" containerName="payload-processing"
Sep 19 12:40:20.080892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080868    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239bffbe-d513-4707-bba3-bd412ff8d3ad" containerName="maas-api"
Sep 19 12:40:20.080892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080876    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="239bffbe-d513-4707-bba3-bd412ff8d3ad" containerName="maas-api"
Sep 19 12:40:20.080989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080941    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="239bffbe-d513-4707-bba3-bd412ff8d3ad" containerName="maas-api"
Sep 19 12:40:20.080989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.080954    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="78c34e1c-be23-4cc5-b5c4-0745a8f983a4" containerName="payload-processing"
Sep 19 12:40:20.085451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.085432    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.088198 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.088174    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 12:40:20.088318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.088175    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-21b2975e-openshift-default-dockercfg-xxmtt\""
Sep 19 12:40:20.096366 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.094584    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"]
Sep 19 12:40:20.186674 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186638    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-credential-socket\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.186842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186683    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gm8ql\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-kube-api-access-gm8ql\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.186842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186719    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-istio-token\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.186842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186786    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-socket\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.186842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186830    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-certs\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.187027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186851    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9454c6ba-7317-4192-b45d-80943f691ce1-istio-podinfo\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.187027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186871    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9454c6ba-7317-4192-b45d-80943f691ce1-istiod-ca-cert\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.187027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186893    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-data\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.187027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.186923    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-envoy\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288308 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288271    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-credential-socket\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288308 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288313    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gm8ql\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-kube-api-access-gm8ql\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288336    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-istio-token\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288363    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-socket\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288400    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-certs\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288427    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9454c6ba-7317-4192-b45d-80943f691ce1-istio-podinfo\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288486    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9454c6ba-7317-4192-b45d-80943f691ce1-istiod-ca-cert\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288513    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-data\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288547    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-envoy\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288799    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-credential-socket\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.288948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.288823    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-socket\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.289056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.289004    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-data\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.289056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.289042    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-certs\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.289311 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.289293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9454c6ba-7317-4192-b45d-80943f691ce1-istiod-ca-cert\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.290809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.290783    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-envoy\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.291134 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.291115    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9454c6ba-7317-4192-b45d-80943f691ce1-istio-podinfo\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.296472 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.296427    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-istio-token\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.296673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.296653    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gm8ql\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-kube-api-access-gm8ql\") pod \"e2e-derive-21b2975e-openshift-default-74768f9944-gblg9\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") " pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.398484 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.398408    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:20.472993 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.472853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:20.673034 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.672954    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"]
Sep 19 12:40:20.676616 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:40:20.676590    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9454c6ba_7317_4192_b45d_80943f691ce1.slice/crio-06409eff7ca9c57ddb2909c171ab28f3b0d4f57700e9ed1ee9b0f007a0c98b59 WatchSource:0}: Error finding container 06409eff7ca9c57ddb2909c171ab28f3b0d4f57700e9ed1ee9b0f007a0c98b59: Status 404 returned error can't find the container with id 06409eff7ca9c57ddb2909c171ab28f3b0d4f57700e9ed1ee9b0f007a0c98b59
Sep 19 12:40:20.679286 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.679249    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:20.679379 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.679321    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:20.679379 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.679349    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:20.812751 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:20.812715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:21.075409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:21.075379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:21.592433 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:21.592399    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" event={"ID":"9454c6ba-7317-4192-b45d-80943f691ce1","Type":"ContainerStarted","Data":"3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"}
Sep 19 12:40:21.592918 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:21.592456    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" event={"ID":"9454c6ba-7317-4192-b45d-80943f691ce1","Type":"ContainerStarted","Data":"06409eff7ca9c57ddb2909c171ab28f3b0d4f57700e9ed1ee9b0f007a0c98b59"}
Sep 19 12:40:21.613119 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:21.613071    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podStartSLOduration=1.6130569380000002 podStartE2EDuration="1.613056938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:21.611244315 +0000 UTC m=+904.971487255" watchObservedRunningTime="2026-09-19 12:40:21.613056938 +0000 UTC m=+904.973299878"
Sep 19 12:40:21.783951 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:21.783901    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:22.398638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.398605    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:22.403681 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.403659    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:22.486148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.486118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:22.524638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.524606    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"]
Sep 19 12:40:22.524871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.524840    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n" podUID="17589e31-e96b-4df8-a805-d4a17a6bbf56" containerName="payload-pre-processing" containerID="cri-o://3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10" gracePeriod=30
Sep 19 12:40:22.586306 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.586274    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:22.597659 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.597637    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:22.598541 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.598522    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:22.690132 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.690039    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:22.777786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.777759    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:40:22.786586 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.786548    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:22.811061 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.811037    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17589e31-e96b-4df8-a805-d4a17a6bbf56-plugins-config-volume\") pod \"17589e31-e96b-4df8-a805-d4a17a6bbf56\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") "
Sep 19 12:40:22.811199 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.811086    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58967\" (UniqueName: \"kubernetes.io/projected/17589e31-e96b-4df8-a805-d4a17a6bbf56-kube-api-access-58967\") pod \"17589e31-e96b-4df8-a805-d4a17a6bbf56\" (UID: \"17589e31-e96b-4df8-a805-d4a17a6bbf56\") "
Sep 19 12:40:22.811455 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.811426    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17589e31-e96b-4df8-a805-d4a17a6bbf56-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "17589e31-e96b-4df8-a805-d4a17a6bbf56" (UID: "17589e31-e96b-4df8-a805-d4a17a6bbf56"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:40:22.813132 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.813111    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17589e31-e96b-4df8-a805-d4a17a6bbf56-kube-api-access-58967" (OuterVolumeSpecName: "kube-api-access-58967") pod "17589e31-e96b-4df8-a805-d4a17a6bbf56" (UID: "17589e31-e96b-4df8-a805-d4a17a6bbf56"). InnerVolumeSpecName "kube-api-access-58967". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:22.912007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.911976    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17589e31-e96b-4df8-a805-d4a17a6bbf56-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:22.912007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.912002    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58967\" (UniqueName: \"kubernetes.io/projected/17589e31-e96b-4df8-a805-d4a17a6bbf56-kube-api-access-58967\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:22.959020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:22.958989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:23.050724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.050687    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:23.149352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.149315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:23.604231 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.604191    2571 generic.go:356] "Generic (PLEG): container finished" podID="17589e31-e96b-4df8-a805-d4a17a6bbf56" containerID="3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10" exitCode=0
Sep 19 12:40:23.604671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.604260    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"
Sep 19 12:40:23.604671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.604279    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n" event={"ID":"17589e31-e96b-4df8-a805-d4a17a6bbf56","Type":"ContainerDied","Data":"3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10"}
Sep 19 12:40:23.604671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.604362    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n" event={"ID":"17589e31-e96b-4df8-a805-d4a17a6bbf56","Type":"ContainerDied","Data":"3b76c88b734f7985d5a764974a19497320ee35cbbe60373206b4b10495d66f54"}
Sep 19 12:40:23.604671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.604385    2571 scope.go:117] "RemoveContainer" containerID="3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10"
Sep 19 12:40:23.613266 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.613251    2571 scope.go:117] "RemoveContainer" containerID="3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10"
Sep 19 12:40:23.613577 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:40:23.613559    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10\": container with ID starting with 3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10 not found: ID does not exist" containerID="3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10"
Sep 19 12:40:23.613616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.613585    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10"} err="failed to get container status \"3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10\": rpc error: code = NotFound desc = could not find container \"3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10\": container with ID starting with 3ddfbd77346ace77f6bcb0b046a76d6dad1d30d9d37041911559a1dfaa019e10 not found: ID does not exist"
Sep 19 12:40:23.625983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.625950    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"]
Sep 19 12:40:23.631633 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.631611    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-38aa0a-f6bf585c8-r796n"]
Sep 19 12:40:23.794374 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:23.794340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:24.565980 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:24.565950    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:24.740141 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:24.740114    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17589e31-e96b-4df8-a805-d4a17a6bbf56" path="/var/lib/kubelet/pods/17589e31-e96b-4df8-a805-d4a17a6bbf56/volumes"
Sep 19 12:40:25.200886 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:25.200849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:25.843971 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:25.843933    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:26.525942 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:26.525908    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:40:26.784680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:26.784582    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:26.885875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:26.885837    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:27.002156 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:27.002093    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:27.241365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:27.241328    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:27.338630 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:27.338597    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:27.614017 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:27.613918    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:28.106185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:28.106155    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"]
Sep 19 12:40:28.150232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:28.150204    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:28.336382 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:28.336346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:30.848781 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:30.848743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:32.414799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.414762    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"]
Sep 19 12:40:32.415253 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.415236    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17589e31-e96b-4df8-a805-d4a17a6bbf56" containerName="payload-pre-processing"
Sep 19 12:40:32.415305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.415257    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="17589e31-e96b-4df8-a805-d4a17a6bbf56" containerName="payload-pre-processing"
Sep 19 12:40:32.415385 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.415375    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="17589e31-e96b-4df8-a805-d4a17a6bbf56" containerName="payload-pre-processing"
Sep 19 12:40:32.419658 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.419640    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.422124 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.422103    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-61583b\"/\"openshift-service-ca.crt\""
Sep 19 12:40:32.423442 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.423424    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-61583b\"/\"test-model-61583b-kserve-self-signed-certs\""
Sep 19 12:40:32.423568 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.423449    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-61583b\"/\"default-dockercfg-wrckm\""
Sep 19 12:40:32.423568 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.423516    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-61583b\"/\"kube-root-ca.crt\""
Sep 19 12:40:32.427471 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.427436    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"]
Sep 19 12:40:32.490811 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.490786    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-home\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.490933 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.490823    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-dshm\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.490933 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.490864    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-model-cache\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.490933 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.490922    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tls-certs\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.491085 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.490980    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tmp-dir\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.491085 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.491004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq4nf\" (UniqueName: \"kubernetes.io/projected/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-kube-api-access-nq4nf\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591514 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591488    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-model-cache\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591539    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tls-certs\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591589    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tmp-dir\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591615    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq4nf\" (UniqueName: \"kubernetes.io/projected/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-kube-api-access-nq4nf\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591660    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-home\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591783 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591692    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-dshm\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.591934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591912    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-model-cache\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.592000 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.591926    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tmp-dir\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.592052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.592005    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-home\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.593839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.593820    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-dshm\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.593982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.593967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tls-certs\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.599203 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.599183    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq4nf\" (UniqueName: \"kubernetes.io/projected/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-kube-api-access-nq4nf\") pod \"test-model-61583b-kserve-9fb6f86c5-pkjnl\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") " pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.730701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.730677    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:32.813123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:32.813069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:33.002603 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:33.002579    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"]
Sep 19 12:40:33.003824 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:40:33.003800    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod009e7b72_924d_4291_8acc_5a2b3a7bb0e5.slice/crio-e538016f259055573adcf2afdd195cea8a7bb6df96c510e78bce485403628044 WatchSource:0}: Error finding container e538016f259055573adcf2afdd195cea8a7bb6df96c510e78bce485403628044: Status 404 returned error can't find the container with id e538016f259055573adcf2afdd195cea8a7bb6df96c510e78bce485403628044
Sep 19 12:40:33.653902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:33.653873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl" event={"ID":"009e7b72-924d-4291-8acc-5a2b3a7bb0e5","Type":"ContainerStarted","Data":"116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b"}
Sep 19 12:40:33.654268 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:33.653911    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl" event={"ID":"009e7b72-924d-4291-8acc-5a2b3a7bb0e5","Type":"ContainerStarted","Data":"e538016f259055573adcf2afdd195cea8a7bb6df96c510e78bce485403628044"}
Sep 19 12:40:33.671725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:33.671681    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl" podStartSLOduration=1.671667263 podStartE2EDuration="1.671667263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:40:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:40:33.669642391 +0000 UTC m=+917.029885333" watchObservedRunningTime="2026-09-19 12:40:33.671667263 +0000 UTC m=+917.031910203"
Sep 19 12:40:42.731368 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:42.731335    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:42.731977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:42.731415    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:42.768624 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:42.768590    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:43.125283 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.125191    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr" podUID="77118fd8-2e1c-4734-840d-ec915f4fc149" containerName="main" containerID="cri-o://33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518" gracePeriod=45
Sep 19 12:40:43.402116 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.402092    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:40:43.493745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493718    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-tmp-dir\") pod \"77118fd8-2e1c-4734-840d-ec915f4fc149\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") "
Sep 19 12:40:43.493940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493756    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8pvw\" (UniqueName: \"kubernetes.io/projected/77118fd8-2e1c-4734-840d-ec915f4fc149-kube-api-access-s8pvw\") pod \"77118fd8-2e1c-4734-840d-ec915f4fc149\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") "
Sep 19 12:40:43.493940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493773    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-model-cache\") pod \"77118fd8-2e1c-4734-840d-ec915f4fc149\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") "
Sep 19 12:40:43.493940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493815    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77118fd8-2e1c-4734-840d-ec915f4fc149-tls-certs\") pod \"77118fd8-2e1c-4734-840d-ec915f4fc149\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") "
Sep 19 12:40:43.493940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493835    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-dshm\") pod \"77118fd8-2e1c-4734-840d-ec915f4fc149\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") "
Sep 19 12:40:43.493940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493848    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-home\") pod \"77118fd8-2e1c-4734-840d-ec915f4fc149\" (UID: \"77118fd8-2e1c-4734-840d-ec915f4fc149\") "
Sep 19 12:40:43.494227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.493981    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77118fd8-2e1c-4734-840d-ec915f4fc149" (UID: "77118fd8-2e1c-4734-840d-ec915f4fc149"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:43.494227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.494072    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-model-cache" (OuterVolumeSpecName: "model-cache") pod "77118fd8-2e1c-4734-840d-ec915f4fc149" (UID: "77118fd8-2e1c-4734-840d-ec915f4fc149"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:43.494227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.494134    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-home" (OuterVolumeSpecName: "home") pod "77118fd8-2e1c-4734-840d-ec915f4fc149" (UID: "77118fd8-2e1c-4734-840d-ec915f4fc149"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:43.494371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.494274    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:43.494371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.494291    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:43.494371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.494308    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:43.496003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.495977    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77118fd8-2e1c-4734-840d-ec915f4fc149-kube-api-access-s8pvw" (OuterVolumeSpecName: "kube-api-access-s8pvw") pod "77118fd8-2e1c-4734-840d-ec915f4fc149" (UID: "77118fd8-2e1c-4734-840d-ec915f4fc149"). InnerVolumeSpecName "kube-api-access-s8pvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:40:43.496098 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.496000    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-dshm" (OuterVolumeSpecName: "dshm") pod "77118fd8-2e1c-4734-840d-ec915f4fc149" (UID: "77118fd8-2e1c-4734-840d-ec915f4fc149"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:40:43.496192 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.496174    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77118fd8-2e1c-4734-840d-ec915f4fc149-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77118fd8-2e1c-4734-840d-ec915f4fc149" (UID: "77118fd8-2e1c-4734-840d-ec915f4fc149"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:40:43.595288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.595253    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77118fd8-2e1c-4734-840d-ec915f4fc149-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:43.595288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.595280    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77118fd8-2e1c-4734-840d-ec915f4fc149-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:43.595288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.595289    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8pvw\" (UniqueName: \"kubernetes.io/projected/77118fd8-2e1c-4734-840d-ec915f4fc149-kube-api-access-s8pvw\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:40:43.716482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.716426    2571 generic.go:356] "Generic (PLEG): container finished" podID="77118fd8-2e1c-4734-840d-ec915f4fc149" containerID="33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518" exitCode=0
Sep 19 12:40:43.716663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.716506    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"
Sep 19 12:40:43.716663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.716516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr" event={"ID":"77118fd8-2e1c-4734-840d-ec915f4fc149","Type":"ContainerDied","Data":"33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518"}
Sep 19 12:40:43.716663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.716570    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr" event={"ID":"77118fd8-2e1c-4734-840d-ec915f4fc149","Type":"ContainerDied","Data":"7ee7f79eaeaaf4b184bf10a420cb73ac72a7920dfb0aa7a8b4feaff0102027b3"}
Sep 19 12:40:43.716663 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.716591    2571 scope.go:117] "RemoveContainer" containerID="33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518"
Sep 19 12:40:43.726102 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.726087    2571 scope.go:117] "RemoveContainer" containerID="33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518"
Sep 19 12:40:43.726364 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:40:43.726344    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518\": container with ID starting with 33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518 not found: ID does not exist" containerID="33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518"
Sep 19 12:40:43.726431 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.726384    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518"} err="failed to get container status \"33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518\": rpc error: code = NotFound desc = could not find container \"33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518\": container with ID starting with 33554be9280402e7dd1aafb6d4841ada93b19e26113cddfb8ba5718d0c5f6518 not found: ID does not exist"
Sep 19 12:40:43.728959 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.728939    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:40:43.758481 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.740188    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"]
Sep 19 12:40:43.758481 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:43.743772    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-autores-38aa0a/autores-model-38aa0a-kserve-6fcfcc446c-8nfcr"]
Sep 19 12:40:44.742700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:44.742660    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77118fd8-2e1c-4734-840d-ec915f4fc149" path="/var/lib/kubelet/pods/77118fd8-2e1c-4734-840d-ec915f4fc149/volumes"
Sep 19 12:40:48.742003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:48.741973    2571 generic.go:356] "Generic (PLEG): container finished" podID="9454c6ba-7317-4192-b45d-80943f691ce1" containerID="3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e" exitCode=0
Sep 19 12:40:48.742444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:48.742053    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" event={"ID":"9454c6ba-7317-4192-b45d-80943f691ce1","Type":"ContainerDied","Data":"3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"}
Sep 19 12:40:48.742444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:48.742398    2571 scope.go:117] "RemoveContainer" containerID="3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"
Sep 19 12:40:48.743496 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:48.743447    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:48.743618 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:48.743547    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:48.743618 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:48.743595    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:40:49.053741 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:49.053650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:49.757484 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:49.757428    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" event={"ID":"9454c6ba-7317-4192-b45d-80943f691ce1","Type":"ContainerStarted","Data":"54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0"}
Sep 19 12:40:50.021248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:50.021172    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:40:50.352355 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:50.352251    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:50.399016 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:50.398980    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:50.400155 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:50.400130    2571 patch_prober.go:28] interesting pod/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 19 12:40:50.400292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:50.400198    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 19 12:40:51.063942 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:51.063899    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:51.399717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:51.399634    2571 patch_prober.go:28] interesting pod/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 19 12:40:51.399717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:51.399687    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 19 12:40:51.822082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:51.822047    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:52.399366 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:52.399334    2571 patch_prober.go:28] interesting pod/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 19 12:40:52.399781 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:52.399385    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 19 12:40:52.457225 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:52.457190    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:52.597951 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:52.597919    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:53.127899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:53.127867    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:53.399909 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:53.399812    2571 patch_prober.go:28] interesting pod/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 19 12:40:53.399909 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:53.399885    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 19 12:40:53.973990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:53.973961    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:54.399933 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:54.399826    2571 patch_prober.go:28] interesting pod/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused" start-of-body=
Sep 19 12:40:54.400398 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:54.400305    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": dial tcp 10.134.0.46:15021: connect: connection refused"
Sep 19 12:40:54.755953 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:54.755921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:55.403251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:55.403171    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:55.404039 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:55.404019    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:40:55.459004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:55.458972    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:56.256906 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:56.256865    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:56.955561 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:56.955517    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:57.733323 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:57.733288    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:40:57.834535 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:40:57.834505    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:04.290885 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:04.290847    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:04.895393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:04.895346    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:04.957983 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:04.957933    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.015058 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.015021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.095236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.095200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.151952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.151863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.224966 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.224927    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"]
Sep 19 12:41:05.225208 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.225179    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" containerID="cri-o://54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0" gracePeriod=30
Sep 19 12:41:05.228329 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.228306    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.293767 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.293739    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.451714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.451681    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.548893 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.548858    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.880925 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.880805    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:05.980510 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:05.980471    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:06.077188 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:06.077155    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:06.191752 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:06.191672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:06.655818 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:06.655784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:07.088561 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:07.088516    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:07.423742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:07.423651    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:07.509641 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:07.509604    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:08.277927 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:08.277890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:08.983758 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:08.983727    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:09.080342 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:09.080307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:09.310374 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:09.310277    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:10.092311 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.092277    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:10.475250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.475223    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:41:10.534050 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534017    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gm8ql\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-kube-api-access-gm8ql\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534081    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9454c6ba-7317-4192-b45d-80943f691ce1-istiod-ca-cert\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534118    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-istio-token\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534143    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-data\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534236 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534202    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-socket\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534493 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534237    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-envoy\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534493 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534277    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9454c6ba-7317-4192-b45d-80943f691ce1-istio-podinfo\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534493 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534334    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-credential-socket\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534493 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534370    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-certs\") pod \"9454c6ba-7317-4192-b45d-80943f691ce1\" (UID: \"9454c6ba-7317-4192-b45d-80943f691ce1\") "
Sep 19 12:41:10.534716 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534564    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-data" (OuterVolumeSpecName: "istio-data") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:41:10.534716 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534579    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9454c6ba-7317-4192-b45d-80943f691ce1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:41:10.534833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534714    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:41:10.534833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534778    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:41:10.534833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534785    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9454c6ba-7317-4192-b45d-80943f691ce1-istiod-ca-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.534833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534796    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:41:10.534833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.534819    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-data\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.536439 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.536413    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-istio-token" (OuterVolumeSpecName: "istio-token") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:41:10.536559 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.536436    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-kube-api-access-gm8ql" (OuterVolumeSpecName: "kube-api-access-gm8ql") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "kube-api-access-gm8ql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:41:10.536559 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.536417    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9454c6ba-7317-4192-b45d-80943f691ce1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:41:10.536693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.536673    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9454c6ba-7317-4192-b45d-80943f691ce1" (UID: "9454c6ba-7317-4192-b45d-80943f691ce1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.635954    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.635989    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-istio-envoy\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.636000    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9454c6ba-7317-4192-b45d-80943f691ce1-istio-podinfo\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.636011    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-credential-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.636022    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9454c6ba-7317-4192-b45d-80943f691ce1-workload-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.636034    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gm8ql\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-kube-api-access-gm8ql\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.636053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.636046    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9454c6ba-7317-4192-b45d-80943f691ce1-istio-token\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:41:10.858932 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.858893    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:10.863113 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.863086    2571 generic.go:356] "Generic (PLEG): container finished" podID="9454c6ba-7317-4192-b45d-80943f691ce1" containerID="54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0" exitCode=0
Sep 19 12:41:10.863245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.863126    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"
Sep 19 12:41:10.863245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.863169    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" event={"ID":"9454c6ba-7317-4192-b45d-80943f691ce1","Type":"ContainerDied","Data":"54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0"}
Sep 19 12:41:10.863245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.863218    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" event={"ID":"9454c6ba-7317-4192-b45d-80943f691ce1","Type":"ContainerDied","Data":"06409eff7ca9c57ddb2909c171ab28f3b0d4f57700e9ed1ee9b0f007a0c98b59"}
Sep 19 12:41:10.863245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.863241    2571 scope.go:117] "RemoveContainer" containerID="54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0"
Sep 19 12:41:10.871768 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.871748    2571 scope.go:117] "RemoveContainer" containerID="3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"
Sep 19 12:41:10.880596 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.880569    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"]
Sep 19 12:41:10.880860 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.880847    2571 scope.go:117] "RemoveContainer" containerID="54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0"
Sep 19 12:41:10.881133 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:41:10.881114    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0\": container with ID starting with 54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0 not found: ID does not exist" containerID="54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0"
Sep 19 12:41:10.881207 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.881144    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0"} err="failed to get container status \"54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0\": rpc error: code = NotFound desc = could not find container \"54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0\": container with ID starting with 54583cf7262e118aa1d8fdbea78ede777122afc087934de4237a5c28ded6a5e0 not found: ID does not exist"
Sep 19 12:41:10.881207 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.881167    2571 scope.go:117] "RemoveContainer" containerID="3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"
Sep 19 12:41:10.881514 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:41:10.881491    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e\": container with ID starting with 3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e not found: ID does not exist" containerID="3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"
Sep 19 12:41:10.881560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.881523    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e"} err="failed to get container status \"3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e\": rpc error: code = NotFound desc = could not find container \"3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e\": container with ID starting with 3b9ec7ac1395d6422be12ab4fbdf318b98911cc0496ada1efb196de1bc72774e not found: ID does not exist"
Sep 19 12:41:10.884964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:10.884943    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9"]
Sep 19 12:41:11.404388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:11.404344    2571 patch_prober.go:28] interesting pod/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 12:41:11.404586 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:11.404422    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-21b2975e-openshift-default-74768f9944-gblg9" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 12:41:11.559695 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:11.559658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:12.218837 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.218803    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"]
Sep 19 12:41:12.219285 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219259    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77118fd8-2e1c-4734-840d-ec915f4fc149" containerName="main"
Sep 19 12:41:12.219333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219290    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="77118fd8-2e1c-4734-840d-ec915f4fc149" containerName="main"
Sep 19 12:41:12.219333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219310    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy"
Sep 19 12:41:12.219333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219319    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy"
Sep 19 12:41:12.219435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219333    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy"
Sep 19 12:41:12.219435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219342    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy"
Sep 19 12:41:12.219529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219446    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy"
Sep 19 12:41:12.219529 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.219492    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="77118fd8-2e1c-4734-840d-ec915f4fc149" containerName="main"
Sep 19 12:41:12.224255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.224237    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.226607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.226588    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-b4e91c-serving-cert\""
Sep 19 12:41:12.232553 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.231787    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"]
Sep 19 12:41:12.249593 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.249566    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:12.303163 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.303131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:12.351365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.351331    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957fe1e0-3f11-4beb-bb0e-8d1077de7573-openshift-service-ca\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.351545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.351386    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-metrics-certs\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.351545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.351484    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.351545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.351538    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzkqq\" (UniqueName: \"kubernetes.io/projected/957fe1e0-3f11-4beb-bb0e-8d1077de7573-kube-api-access-fzkqq\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.452115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.452083    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzkqq\" (UniqueName: \"kubernetes.io/projected/957fe1e0-3f11-4beb-bb0e-8d1077de7573-kube-api-access-fzkqq\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.452315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.452149    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957fe1e0-3f11-4beb-bb0e-8d1077de7573-openshift-service-ca\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.452315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.452180    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-metrics-certs\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.452315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.452207    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.452511 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:41:12.452316    2571 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-serving-cert: secret "maas-api-e2e-explicit-b4e91c-serving-cert" not found
Sep 19 12:41:12.452511 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:41:12.452384    2571 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls podName:957fe1e0-3f11-4beb-bb0e-8d1077de7573 nodeName:}" failed. No retries permitted until 2026-09-19 12:41:12.952367544 +0000 UTC m=+956.312610463 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls") pod "maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" (UID: "957fe1e0-3f11-4beb-bb0e-8d1077de7573") : secret "maas-api-e2e-explicit-b4e91c-serving-cert" not found
Sep 19 12:41:12.452808 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.452790    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957fe1e0-3f11-4beb-bb0e-8d1077de7573-openshift-service-ca\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.454740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.454723    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-metrics-certs\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.460319 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.460293    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzkqq\" (UniqueName: \"kubernetes.io/projected/957fe1e0-3f11-4beb-bb0e-8d1077de7573-kube-api-access-fzkqq\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.507387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.507298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:12.537651 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.537623    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:41:12.705091 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.705056    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:12.740094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.740064    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" path="/var/lib/kubelet/pods/9454c6ba-7317-4192-b45d-80943f691ce1/volumes"
Sep 19 12:41:12.805067 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.804976    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:12.956893 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.956863    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:12.959216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:12.959197    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls\") pod \"maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:13.137163 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.137075    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:13.406668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.406587    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"]
Sep 19 12:41:13.409358 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:41:13.409336    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod957fe1e0_3f11_4beb_bb0e_8d1077de7573.slice/crio-92c0b99e8f857a072dbda60db48b2347b02749f616f87433605ffaf948c0c7c6 WatchSource:0}: Error finding container 92c0b99e8f857a072dbda60db48b2347b02749f616f87433605ffaf948c0c7c6: Status 404 returned error can't find the container with id 92c0b99e8f857a072dbda60db48b2347b02749f616f87433605ffaf948c0c7c6
Sep 19 12:41:13.410721 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.410704    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 12:41:13.671710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.671632    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:13.784184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.784137    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:13.872476 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.872432    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:13.880978 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.880951    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" event={"ID":"957fe1e0-3f11-4beb-bb0e-8d1077de7573","Type":"ContainerStarted","Data":"92c0b99e8f857a072dbda60db48b2347b02749f616f87433605ffaf948c0c7c6"}
Sep 19 12:41:13.973487 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:13.973450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:14.361850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:14.361811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:14.886935 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:14.886903    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" event={"ID":"957fe1e0-3f11-4beb-bb0e-8d1077de7573","Type":"ContainerStarted","Data":"1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f"}
Sep 19 12:41:14.887373 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:14.887037    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:14.934073 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:14.934025    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" podStartSLOduration=2.435642674 podStartE2EDuration="2.934009808s" totalImagesPullingTime="498.367134ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:41:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:41:14.933003039 +0000 UTC m=+958.293245980" watchObservedRunningTime="2026-09-19 12:41:14.934009808 +0000 UTC m=+958.294252797"
Sep 19 12:41:20.896862 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:20.896828    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:41:30.121321 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:30.121241    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:41:34.753714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:34.753676    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:35.533724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:35.533690    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:36.186736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:36.186692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:36.865878 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:36.865844    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:37.553002 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:37.552954    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:38.291204 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:38.291152    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:38.909661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:38.909631    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:40.238384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:40.238347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:46.033604 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:46.033551    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:46.170281 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:46.170245    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:46.255435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:46.255378    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:50.921644 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:50.921614    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"]
Sep 19 12:41:50.922100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:50.922083    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9454c6ba-7317-4192-b45d-80943f691ce1" containerName="istio-proxy"
Sep 19 12:41:50.925129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:50.925113    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:50.927867 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:50.927845    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"premium-w0-a2e924-kserve-self-signed-certs\""
Sep 19 12:41:50.933488 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:50.933451    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"]
Sep 19 12:41:50.951639 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:50.951610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:41:51.000786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.000757    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-home\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.000939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.000820    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-dshm\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.000939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.000842    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-tmp-dir\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.000939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.000865    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b9066-4012-4ba0-84a4-e37832db3adf-tls-certs\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.000939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.000881    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxl4p\" (UniqueName: \"kubernetes.io/projected/a47b9066-4012-4ba0-84a4-e37832db3adf-kube-api-access-zxl4p\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.000939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.000907    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-model-cache\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102060 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102022    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-dshm\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102060 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102064    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-tmp-dir\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102099    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b9066-4012-4ba0-84a4-e37832db3adf-tls-certs\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102121    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxl4p\" (UniqueName: \"kubernetes.io/projected/a47b9066-4012-4ba0-84a4-e37832db3adf-kube-api-access-zxl4p\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102164    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-model-cache\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102204    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-home\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102554    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-tmp-dir\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102635 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102590    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-model-cache\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.102687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.102632    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-home\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.104417 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.104397    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-dshm\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.104665 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.104646    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b9066-4012-4ba0-84a4-e37832db3adf-tls-certs\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.110101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.110079    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxl4p\" (UniqueName: \"kubernetes.io/projected/a47b9066-4012-4ba0-84a4-e37832db3adf-kube-api-access-zxl4p\") pod \"premium-w0-a2e924-kserve-5c7f655dcc-q8jhw\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") " pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.235413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.235377    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:41:51.503687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:51.503664    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"]
Sep 19 12:41:51.505563 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:41:51.505528    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda47b9066_4012_4ba0_84a4_e37832db3adf.slice/crio-080c6853a65e598d74f02b692abc5d4aaa620e0ac65166f2253957b2f54eaf82 WatchSource:0}: Error finding container 080c6853a65e598d74f02b692abc5d4aaa620e0ac65166f2253957b2f54eaf82: Status 404 returned error can't find the container with id 080c6853a65e598d74f02b692abc5d4aaa620e0ac65166f2253957b2f54eaf82
Sep 19 12:41:52.059573 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:52.059541    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw" event={"ID":"a47b9066-4012-4ba0-84a4-e37832db3adf","Type":"ContainerStarted","Data":"743e14ec9d5f445269515ac106d167d6db950cea4fedd072dfcb191b1d058262"}
Sep 19 12:41:52.059976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:52.059581    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw" event={"ID":"a47b9066-4012-4ba0-84a4-e37832db3adf","Type":"ContainerStarted","Data":"080c6853a65e598d74f02b692abc5d4aaa620e0ac65166f2253957b2f54eaf82"}
Sep 19 12:41:52.077649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:52.077603    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw" podStartSLOduration=2.077587536 podStartE2EDuration="2.077587536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:41:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:41:52.075743212 +0000 UTC m=+995.435986156" watchObservedRunningTime="2026-09-19 12:41:52.077587536 +0000 UTC m=+995.437830477"
Sep 19 12:41:56.536852 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:41:56.536812    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:42:01.236127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:01.236095    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:42:01.236525 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:01.236142    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:42:01.248388 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:01.248367    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:42:02.040621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:02.040583    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:02.118121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:02.118096    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:42:04.419879 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:04.419849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:07.721827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:07.721790    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:07.789614 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:07.789579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:07.862242 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:07.862206    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:07.961627 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:07.961591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:10.109734 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.109696    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"]
Sep 19 12:42:10.112821 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.112805    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.116844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.116817    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5be349\"/\"kube-root-ca.crt\""
Sep 19 12:42:10.116844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.116837    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5be349\"/\"default-dockercfg-7744g\""
Sep 19 12:42:10.117020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.116842    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-5be349\"/\"test-model-5be349-kserve-self-signed-certs\""
Sep 19 12:42:10.117020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.116817    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-5be349\"/\"openshift-service-ca.crt\""
Sep 19 12:42:10.122805 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.122781    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"]
Sep 19 12:42:10.237305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.237271    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:10.274729 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.274703    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tls-certs\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.274871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.274736    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhxh6\" (UniqueName: \"kubernetes.io/projected/e69ebde9-1fe9-461b-9b7c-9110d34388a8-kube-api-access-vhxh6\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.274871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.274770    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-model-cache\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.274871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.274837    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-home\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.274982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.274900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-dshm\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.274982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.274951    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tmp-dir\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376152    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tmp-dir\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376190    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tls-certs\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376214    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhxh6\" (UniqueName: \"kubernetes.io/projected/e69ebde9-1fe9-461b-9b7c-9110d34388a8-kube-api-access-vhxh6\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376565 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376254    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-model-cache\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376565 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376280    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-home\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376565 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376322    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-dshm\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376731 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376666    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tmp-dir\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376731 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376671    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-model-cache\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.376731 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.376715    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-home\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.378494 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.378475    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-dshm\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.378772 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.378755    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tls-certs\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.384092 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.384072    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhxh6\" (UniqueName: \"kubernetes.io/projected/e69ebde9-1fe9-461b-9b7c-9110d34388a8-kube-api-access-vhxh6\") pod \"test-model-5be349-kserve-7b5cb4fd5c-tr8sv\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") " pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.423115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.423075    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:10.692449 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:10.692425    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"]
Sep 19 12:42:10.693530 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:42:10.693507    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode69ebde9_1fe9_461b_9b7c_9110d34388a8.slice/crio-b3982c282c71f521fab48e93c802cc4d8d267c012e2ff28c3473ea30a827e6f2 WatchSource:0}: Error finding container b3982c282c71f521fab48e93c802cc4d8d267c012e2ff28c3473ea30a827e6f2: Status 404 returned error can't find the container with id b3982c282c71f521fab48e93c802cc4d8d267c012e2ff28c3473ea30a827e6f2
Sep 19 12:42:11.150061 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:11.150029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv" event={"ID":"e69ebde9-1fe9-461b-9b7c-9110d34388a8","Type":"ContainerStarted","Data":"d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d"}
Sep 19 12:42:11.150451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:11.150069    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv" event={"ID":"e69ebde9-1fe9-461b-9b7c-9110d34388a8","Type":"ContainerStarted","Data":"b3982c282c71f521fab48e93c802cc4d8d267c012e2ff28c3473ea30a827e6f2"}
Sep 19 12:42:11.166963 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:11.166913    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv" podStartSLOduration=1.166899544 podStartE2EDuration="1.166899544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:42:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:42:11.165799161 +0000 UTC m=+1014.526042102" watchObservedRunningTime="2026-09-19 12:42:11.166899544 +0000 UTC m=+1014.527142486"
Sep 19 12:42:12.826010 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:12.825976    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:42:18.385485 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:18.385431    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:19.131833 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:19.131795    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:20.423335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:20.423295    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:20.423746 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:20.423349    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:20.435567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:20.435543    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:21.212150 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:21.212122    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:42:24.917542 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:24.917510    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:24.973692 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:24.973661    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:25.045504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:25.045476    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:25.139124 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:25.139089    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:25.239914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:25.239882    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:25.799319 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:25.799285    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:25.894420 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:25.894374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:25.995130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:25.995094    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:27.400559 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:27.400530    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:32.006181 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:32.006134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:32.057219 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:32.057181    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:33.213736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:33.213695    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:33.314319 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:33.314275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:33.436379 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:33.436345    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:42:33.844043 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:33.844003    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:39.624426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:39.624389    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:39.685653 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:39.685619    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:39.837823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:39.837772    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:40.119975 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:40.119941    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:40.620318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:40.620279    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:44.865885 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:44.865851    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:45.659518 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:45.659482    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:45.757532 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:45.757496    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:46.481907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:46.481868    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:47.259287 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:47.259256    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:47.944327 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:47.944292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:48.782346 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:48.782299    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:49.464260 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:49.464228    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:49.565512 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:49.565448    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:49.665647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:49.665615    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:49.920507 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:49.920401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:50.025673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.025636    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:42:50.026970 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.026939    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"]
Sep 19 12:42:50.030383 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.030368    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.032914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.032889    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-f9a7df96-dockercfg-ltcrj\""
Sep 19 12:42:50.033031 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.032890    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-f9a7df96\""
Sep 19 12:42:50.038956 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.038933    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"]
Sep 19 12:42:50.061335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.061306    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"]
Sep 19 12:42:50.064479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.064450    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.074450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.074425    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"]
Sep 19 12:42:50.125013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.124985    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8040ac60-3da9-4378-b144-239e86b22dfa-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.125013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.125015    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzbhb\" (UniqueName: \"kubernetes.io/projected/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-kube-api-access-dzbhb\") pod \"payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") " pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.125286 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.125052    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55qjt\" (UniqueName: \"kubernetes.io/projected/8040ac60-3da9-4378-b144-239e86b22dfa-kube-api-access-55qjt\") pod \"payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.125286 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.125204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-plugins-config-volume\") pod \"payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") " pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.225849 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.225815    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-plugins-config-volume\") pod \"payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") " pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.226018 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.225881    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8040ac60-3da9-4378-b144-239e86b22dfa-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.226018 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.225898    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzbhb\" (UniqueName: \"kubernetes.io/projected/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-kube-api-access-dzbhb\") pod \"payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") " pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.226018 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.225930    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55qjt\" (UniqueName: \"kubernetes.io/projected/8040ac60-3da9-4378-b144-239e86b22dfa-kube-api-access-55qjt\") pod \"payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.226587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.226568    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8040ac60-3da9-4378-b144-239e86b22dfa-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.226634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.226594    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-plugins-config-volume\") pod \"payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") " pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.234090 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.234060    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzbhb\" (UniqueName: \"kubernetes.io/projected/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-kube-api-access-dzbhb\") pod \"payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") " pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.234211 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.234132    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55qjt\" (UniqueName: \"kubernetes.io/projected/8040ac60-3da9-4378-b144-239e86b22dfa-kube-api-access-55qjt\") pod \"payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.342101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.342050    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:50.375391 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.375360    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:50.613577 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.613553    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"]
Sep 19 12:42:50.614898 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:42:50.614866    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8040ac60_3da9_4378_b144_239e86b22dfa.slice/crio-093ad84cabe612f764f3fc7fdda3c6a0ea15026b67c7ee9195c1f07f0d4c2556 WatchSource:0}: Error finding container 093ad84cabe612f764f3fc7fdda3c6a0ea15026b67c7ee9195c1f07f0d4c2556: Status 404 returned error can't find the container with id 093ad84cabe612f764f3fc7fdda3c6a0ea15026b67c7ee9195c1f07f0d4c2556
Sep 19 12:42:50.656156 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:50.656131    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"]
Sep 19 12:42:50.657396 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:42:50.657371    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecdddc78_34f9_46ce_ab76_6bc53c7a3ed8.slice/crio-0496559179f110edbddbbf8a5f443aa6278b52c71e242be3a6d43ab44526c9ee WatchSource:0}: Error finding container 0496559179f110edbddbbf8a5f443aa6278b52c71e242be3a6d43ab44526c9ee: Status 404 returned error can't find the container with id 0496559179f110edbddbbf8a5f443aa6278b52c71e242be3a6d43ab44526c9ee
Sep 19 12:42:51.354252 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.354216    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9" event={"ID":"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8","Type":"ContainerStarted","Data":"2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820"}
Sep 19 12:42:51.354477 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.354277    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9" event={"ID":"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8","Type":"ContainerStarted","Data":"0496559179f110edbddbbf8a5f443aa6278b52c71e242be3a6d43ab44526c9ee"}
Sep 19 12:42:51.354477 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.354306    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:42:51.364476 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.364368    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg" event={"ID":"8040ac60-3da9-4378-b144-239e86b22dfa","Type":"ContainerStarted","Data":"2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e"}
Sep 19 12:42:51.364476 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.364453    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg" event={"ID":"8040ac60-3da9-4378-b144-239e86b22dfa","Type":"ContainerStarted","Data":"093ad84cabe612f764f3fc7fdda3c6a0ea15026b67c7ee9195c1f07f0d4c2556"}
Sep 19 12:42:51.364686 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.364500    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:42:51.370623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.370570    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9" podStartSLOduration=1.370552732 podStartE2EDuration="1.370552732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:42:51.370344706 +0000 UTC m=+1054.730587651" watchObservedRunningTime="2026-09-19 12:42:51.370552732 +0000 UTC m=+1054.730795664"
Sep 19 12:42:51.385573 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:42:51.385531    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg" podStartSLOduration=1.3855178129999999 podStartE2EDuration="1.385517813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:42:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:42:51.384540499 +0000 UTC m=+1054.744783441" watchObservedRunningTime="2026-09-19 12:42:51.385517813 +0000 UTC m=+1054.745760754"
Sep 19 12:43:02.371568 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:02.371492    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:43:02.371568 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:02.371564    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:43:03.823608 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:03.823572    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:43:10.145493 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:10.145442    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:12.950570 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:12.950535    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:13.651362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:13.651323    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:14.399824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:14.399791    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:15.101844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:15.101806    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:15.618481 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:15.618426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:15.832178 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:15.832140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:16.233627 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:16.233590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:16.333712 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:16.333678    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:16.656844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:16.656749    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:16.758787 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:16.758752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:17.528669 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:17.528629    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:22.333530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:22.333498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:22.434284 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:22.434250    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:23.345813 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:23.345770    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:24.270158 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:24.270121    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:24.338479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:24.338427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:43:24.991802 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:24.991758    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:25.775330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:25.775299    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:26.565111 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:26.565076    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:27.342089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:27.342050    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:28.240377 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:28.240343    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:31.887734 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:31.887692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:32.790617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:32.790579    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:33.444638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:33.444602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:33.996842 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:33.996815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:34.032298 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.032268    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"]
Sep 19 12:43:34.032530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.032502    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9" podUID="ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" containerName="payload-processing" containerID="cri-o://2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820" gracePeriod=30
Sep 19 12:43:34.038499 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.038448    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"]
Sep 19 12:43:34.038714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.038689    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg" podUID="8040ac60-3da9-4378-b144-239e86b22dfa" containerName="payload-pre-processing" containerID="cri-o://2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e" gracePeriod=30
Sep 19 12:43:34.075318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.075288    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:34.139745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.139711    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:34.201169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.201095    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:34.301994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.301974    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:43:34.304864 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.304847    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:43:34.429627 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.429594    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8040ac60-3da9-4378-b144-239e86b22dfa-plugins-config-volume\") pod \"8040ac60-3da9-4378-b144-239e86b22dfa\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") "
Sep 19 12:43:34.429841 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.429637    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzbhb\" (UniqueName: \"kubernetes.io/projected/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-kube-api-access-dzbhb\") pod \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") "
Sep 19 12:43:34.429841 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.429690    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55qjt\" (UniqueName: \"kubernetes.io/projected/8040ac60-3da9-4378-b144-239e86b22dfa-kube-api-access-55qjt\") pod \"8040ac60-3da9-4378-b144-239e86b22dfa\" (UID: \"8040ac60-3da9-4378-b144-239e86b22dfa\") "
Sep 19 12:43:34.429841 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.429728    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-plugins-config-volume\") pod \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\" (UID: \"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8\") "
Sep 19 12:43:34.430127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.430099    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8040ac60-3da9-4378-b144-239e86b22dfa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8040ac60-3da9-4378-b144-239e86b22dfa" (UID: "8040ac60-3da9-4378-b144-239e86b22dfa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:34.430176 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.430163    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" (UID: "ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:34.431849 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.431819    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8040ac60-3da9-4378-b144-239e86b22dfa-kube-api-access-55qjt" (OuterVolumeSpecName: "kube-api-access-55qjt") pod "8040ac60-3da9-4378-b144-239e86b22dfa" (UID: "8040ac60-3da9-4378-b144-239e86b22dfa"). InnerVolumeSpecName "kube-api-access-55qjt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:34.431975 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.431953    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-kube-api-access-dzbhb" (OuterVolumeSpecName: "kube-api-access-dzbhb") pod "ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" (UID: "ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8"). InnerVolumeSpecName "kube-api-access-dzbhb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:34.450579 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.450553    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:34.530788 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.530706    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55qjt\" (UniqueName: \"kubernetes.io/projected/8040ac60-3da9-4378-b144-239e86b22dfa-kube-api-access-55qjt\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:34.530788 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.530731    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:34.530788 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.530743    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8040ac60-3da9-4378-b144-239e86b22dfa-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:34.530788 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.530755    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzbhb\" (UniqueName: \"kubernetes.io/projected/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8-kube-api-access-dzbhb\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:34.550326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.550292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:34.577775 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.577746    2571 generic.go:356] "Generic (PLEG): container finished" podID="ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" containerID="2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820" exitCode=0
Sep 19 12:43:34.577876 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.577792    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"
Sep 19 12:43:34.577876 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.577833    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9" event={"ID":"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8","Type":"ContainerDied","Data":"2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820"}
Sep 19 12:43:34.577964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.577888    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9" event={"ID":"ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8","Type":"ContainerDied","Data":"0496559179f110edbddbbf8a5f443aa6278b52c71e242be3a6d43ab44526c9ee"}
Sep 19 12:43:34.577964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.577917    2571 scope.go:117] "RemoveContainer" containerID="2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820"
Sep 19 12:43:34.587693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.579552    2571 generic.go:356] "Generic (PLEG): container finished" podID="8040ac60-3da9-4378-b144-239e86b22dfa" containerID="2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e" exitCode=0
Sep 19 12:43:34.587693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.579660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg" event={"ID":"8040ac60-3da9-4378-b144-239e86b22dfa","Type":"ContainerDied","Data":"2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e"}
Sep 19 12:43:34.587693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.579762    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg" event={"ID":"8040ac60-3da9-4378-b144-239e86b22dfa","Type":"ContainerDied","Data":"093ad84cabe612f764f3fc7fdda3c6a0ea15026b67c7ee9195c1f07f0d4c2556"}
Sep 19 12:43:34.587693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.579859    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"
Sep 19 12:43:34.593786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.593766    2571 scope.go:117] "RemoveContainer" containerID="2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820"
Sep 19 12:43:34.594093 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:43:34.594075    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820\": container with ID starting with 2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820 not found: ID does not exist" containerID="2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820"
Sep 19 12:43:34.594164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.594105    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820"} err="failed to get container status \"2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820\": rpc error: code = NotFound desc = could not find container \"2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820\": container with ID starting with 2c7b7a0c536bfed9c1a5221dbfba832e7b0bfdee74f09cf23a6aa285f842e820 not found: ID does not exist"
Sep 19 12:43:34.594164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.594128    2571 scope.go:117] "RemoveContainer" containerID="2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e"
Sep 19 12:43:34.602546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.602516    2571 scope.go:117] "RemoveContainer" containerID="2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e"
Sep 19 12:43:34.602799 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:43:34.602782    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e\": container with ID starting with 2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e not found: ID does not exist" containerID="2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e"
Sep 19 12:43:34.602853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.602805    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e"} err="failed to get container status \"2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e\": rpc error: code = NotFound desc = could not find container \"2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e\": container with ID starting with 2c6db03a731c7f97ec8417bfba972b2017168aa6a6b1aaee3da7f7f1eed1d94e not found: ID does not exist"
Sep 19 12:43:34.603582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.603565    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"]
Sep 19 12:43:34.607059 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.607038    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-f9a7df96-7ddf4b9fc7-qxnw9"]
Sep 19 12:43:34.616622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.616604    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"]
Sep 19 12:43:34.625050 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.625030    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-f9a7df96-8b8c86758-t5xzg"]
Sep 19 12:43:34.742478 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.742430    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8040ac60-3da9-4378-b144-239e86b22dfa" path="/var/lib/kubelet/pods/8040ac60-3da9-4378-b144-239e86b22dfa/volumes"
Sep 19 12:43:34.742902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:34.742880    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" path="/var/lib/kubelet/pods/ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8/volumes"
Sep 19 12:43:35.125706 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:35.125669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:35.962062 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:35.962029    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:36.761749 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:36.761700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:37.546315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:37.546281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:38.187181 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:38.187136    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:38.849037 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:38.849007    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:39.605788 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:39.605749    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:40.419187 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:40.419140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:40.481513 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:40.481450    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:40.664870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:40.664835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:40.987198 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:40.987167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:41.079237 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:41.079210    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:41.271641 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:41.271575    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:42.353340 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:42.353301    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:43.246948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:43.246915    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:44.046510 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:44.046449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:44.853976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:44.853935    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:45.662611 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:45.662576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:46.451727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:46.451689    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:46.955534 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:46.955496    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:47.014863 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:47.014828    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:47.114087 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:47.114052    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:47.211668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:47.211572    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:47.311406 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:47.311369    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:48.971901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:48.971864    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:43:52.561451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.561423    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"]
Sep 19 12:43:52.561957 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.561692    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" podUID="957fe1e0-3f11-4beb-bb0e-8d1077de7573" containerName="maas-api" containerID="cri-o://1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f" gracePeriod=30
Sep 19 12:43:52.589369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.589339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:52.690631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.690586    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:52.770102 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.770067    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:52.809039 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.809020    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:43:52.859329 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.859256    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:52.992017 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.991991    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls\") pod \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") "
Sep 19 12:43:52.992191 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.992054    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzkqq\" (UniqueName: \"kubernetes.io/projected/957fe1e0-3f11-4beb-bb0e-8d1077de7573-kube-api-access-fzkqq\") pod \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") "
Sep 19 12:43:52.992191 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.992112    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957fe1e0-3f11-4beb-bb0e-8d1077de7573-openshift-service-ca\") pod \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") "
Sep 19 12:43:52.992191 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.992142    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-metrics-certs\") pod \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\" (UID: \"957fe1e0-3f11-4beb-bb0e-8d1077de7573\") "
Sep 19 12:43:52.992474 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.992428    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/957fe1e0-3f11-4beb-bb0e-8d1077de7573-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "957fe1e0-3f11-4beb-bb0e-8d1077de7573" (UID: "957fe1e0-3f11-4beb-bb0e-8d1077de7573"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:43:52.994270 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.994246    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "957fe1e0-3f11-4beb-bb0e-8d1077de7573" (UID: "957fe1e0-3f11-4beb-bb0e-8d1077de7573"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:43:52.994270 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.994255    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/957fe1e0-3f11-4beb-bb0e-8d1077de7573-kube-api-access-fzkqq" (OuterVolumeSpecName: "kube-api-access-fzkqq") pod "957fe1e0-3f11-4beb-bb0e-8d1077de7573" (UID: "957fe1e0-3f11-4beb-bb0e-8d1077de7573"). InnerVolumeSpecName "kube-api-access-fzkqq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:43:52.994414 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:52.994267    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "957fe1e0-3f11-4beb-bb0e-8d1077de7573" (UID: "957fe1e0-3f11-4beb-bb0e-8d1077de7573"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:43:53.093714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.093686    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzkqq\" (UniqueName: \"kubernetes.io/projected/957fe1e0-3f11-4beb-bb0e-8d1077de7573-kube-api-access-fzkqq\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.093714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.093711    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/957fe1e0-3f11-4beb-bb0e-8d1077de7573-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.093873 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.093722    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.093873 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.093731    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/957fe1e0-3f11-4beb-bb0e-8d1077de7573-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:43:53.113354 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.113287    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:53.257022 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.256997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:53.500589 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.500557    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:53.601323 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.601284    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:53.679671 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.679635    2571 generic.go:356] "Generic (PLEG): container finished" podID="957fe1e0-3f11-4beb-bb0e-8d1077de7573" containerID="1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f" exitCode=0
Sep 19 12:43:53.679824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.679678    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"
Sep 19 12:43:53.679824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.679718    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" event={"ID":"957fe1e0-3f11-4beb-bb0e-8d1077de7573","Type":"ContainerDied","Data":"1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f"}
Sep 19 12:43:53.679824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.679773    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd" event={"ID":"957fe1e0-3f11-4beb-bb0e-8d1077de7573","Type":"ContainerDied","Data":"92c0b99e8f857a072dbda60db48b2347b02749f616f87433605ffaf948c0c7c6"}
Sep 19 12:43:53.679824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.679795    2571 scope.go:117] "RemoveContainer" containerID="1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f"
Sep 19 12:43:53.689186 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.689171    2571 scope.go:117] "RemoveContainer" containerID="1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f"
Sep 19 12:43:53.689483 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:43:53.689448    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f\": container with ID starting with 1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f not found: ID does not exist" containerID="1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f"
Sep 19 12:43:53.689575 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.689492    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f"} err="failed to get container status \"1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f\": rpc error: code = NotFound desc = could not find container \"1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f\": container with ID starting with 1ee7f6faa5f4773fdd01d7848f930d081024fd1f9f25b7b8c758164d684d139f not found: ID does not exist"
Sep 19 12:43:53.700740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.700717    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"]
Sep 19 12:43:53.706371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.706348    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b4e91c-6d56599c5f-z8xnd"]
Sep 19 12:43:53.978122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:53.978093    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:54.369659 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:54.369578    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:54.658640 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:54.658555    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:54.743422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:54.743379    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="957fe1e0-3f11-4beb-bb0e-8d1077de7573" path="/var/lib/kubelet/pods/957fe1e0-3f11-4beb-bb0e-8d1077de7573/volumes"
Sep 19 12:43:54.975512 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:54.975455    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:55.075965 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:55.075921    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:55.756197 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:55.756157    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:56.669232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:56.669189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:57.466878 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:57.466835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:57.823302 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:57.823220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:57.915813 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:57.915779    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:58.133730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:58.133644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:58.276004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:58.275967    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:59.163093 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:59.163055    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:43:59.858703 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:43:59.858672    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:00.654693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:00.654652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:01.365191 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:01.365163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:01.466311 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:01.466270    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:01.891299 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:01.891259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:01.993500 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:01.993435    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:02.034776 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.034744    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"]
Sep 19 12:44:02.035097 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035084    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" containerName="payload-processing"
Sep 19 12:44:02.035097 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035098    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" containerName="payload-processing"
Sep 19 12:44:02.035185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035107    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="957fe1e0-3f11-4beb-bb0e-8d1077de7573" containerName="maas-api"
Sep 19 12:44:02.035185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035113    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="957fe1e0-3f11-4beb-bb0e-8d1077de7573" containerName="maas-api"
Sep 19 12:44:02.035185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035121    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8040ac60-3da9-4378-b144-239e86b22dfa" containerName="payload-pre-processing"
Sep 19 12:44:02.035185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035126    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="8040ac60-3da9-4378-b144-239e86b22dfa" containerName="payload-pre-processing"
Sep 19 12:44:02.035185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035180    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="8040ac60-3da9-4378-b144-239e86b22dfa" containerName="payload-pre-processing"
Sep 19 12:44:02.035354 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035189    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="957fe1e0-3f11-4beb-bb0e-8d1077de7573" containerName="maas-api"
Sep 19 12:44:02.035354 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.035197    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ecdddc78-34f9-46ce-ab76-6bc53c7a3ed8" containerName="payload-processing"
Sep 19 12:44:02.039613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.039596    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.042305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.042286    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-2a487c-dockercfg-b4h9s\""
Sep 19 12:44:02.042417 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.042286    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-2a487c\""
Sep 19 12:44:02.048041 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.048015    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"]
Sep 19 12:44:02.065017 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.064984    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.065128 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.065059    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cpjl\" (UniqueName: \"kubernetes.io/projected/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-kube-api-access-2cpjl\") pod \"payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.166340 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.166256    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.166340 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.166302    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cpjl\" (UniqueName: \"kubernetes.io/projected/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-kube-api-access-2cpjl\") pod \"payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.166982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.166962    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.173889 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.173861    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cpjl\" (UniqueName: \"kubernetes.io/projected/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-kube-api-access-2cpjl\") pod \"payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.350806 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.350763    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.619401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.619371    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"]
Sep 19 12:44:02.621113 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:44:02.621087    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d2a54cc_5e0a_4cdf_9ac5_dc0a38c1c737.slice/crio-557ff9fe24ba09dfaf82360a9d619de97bb878e2f05b6282f59fc375fcfb019b WatchSource:0}: Error finding container 557ff9fe24ba09dfaf82360a9d619de97bb878e2f05b6282f59fc375fcfb019b: Status 404 returned error can't find the container with id 557ff9fe24ba09dfaf82360a9d619de97bb878e2f05b6282f59fc375fcfb019b
Sep 19 12:44:02.726786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.726752    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r" event={"ID":"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737","Type":"ContainerStarted","Data":"b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb"}
Sep 19 12:44:02.726955 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.726807    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r" event={"ID":"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737","Type":"ContainerStarted","Data":"557ff9fe24ba09dfaf82360a9d619de97bb878e2f05b6282f59fc375fcfb019b"}
Sep 19 12:44:02.726955 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.726828    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:02.744352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:02.744307    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r" podStartSLOduration=0.744293543 podStartE2EDuration="744.293543ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:44:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:44:02.743908011 +0000 UTC m=+1126.104150953" watchObservedRunningTime="2026-09-19 12:44:02.744293543 +0000 UTC m=+1126.104536483"
Sep 19 12:44:06.482945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:06.482908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:13.735559 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:13.735528    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:44:17.943343 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:17.943305    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:24.889395 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:24.889362    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:24.951749 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:24.951717    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:44:25.880843 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:25.880809    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:26.596025 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:26.595990    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:27.450169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:27.450133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:27.848136 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:27.848048    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:28.156499 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:28.156401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:28.257505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:28.257444    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:28.480350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:28.480320    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:28.640771 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:28.640741    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:28.951437 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:28.951391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:31.333781 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:31.333688    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:32.055536 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:32.055499    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:32.153709 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:32.153658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:32.950733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:32.950698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:33.718612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:33.718576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:34.559056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:34.559024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:35.101327 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:35.101294    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:35.268753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:35.268725    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:35.366949 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:35.366857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:35.468227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:35.468189    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:35.821558 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:35.821520    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.091609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.091521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.295984 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.295917    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.396063 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.395969    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.526775 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.526738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.627173 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.627140    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.815451 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.815416    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.914634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.914599    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:36.921214 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.921187    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"]
Sep 19 12:44:36.924398 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.924382    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:36.927007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.926986    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-5068d9\""
Sep 19 12:44:36.927142 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.927022    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-5068d9-dockercfg-h8x89\""
Sep 19 12:44:36.938787 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.938744    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"]
Sep 19 12:44:36.947354 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.947322    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"]
Sep 19 12:44:36.951203 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.951183    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:36.962488 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:36.962451    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"]
Sep 19 12:44:37.083952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.083858    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hb9lb\" (UniqueName: \"kubernetes.io/projected/6021844b-979f-42bf-b291-e21a7568aad8-kube-api-access-hb9lb\") pod \"payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.083952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.083900    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/786a0536-42fe-430f-a65e-68e76502c588-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.083952 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.083926    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfsvx\" (UniqueName: \"kubernetes.io/projected/786a0536-42fe-430f-a65e-68e76502c588-kube-api-access-mfsvx\") pod \"payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.084167 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.084067    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6021844b-979f-42bf-b291-e21a7568aad8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.185372 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.185337    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hb9lb\" (UniqueName: \"kubernetes.io/projected/6021844b-979f-42bf-b291-e21a7568aad8-kube-api-access-hb9lb\") pod \"payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.185372 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.185375    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/786a0536-42fe-430f-a65e-68e76502c588-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.185609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.185393    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfsvx\" (UniqueName: \"kubernetes.io/projected/786a0536-42fe-430f-a65e-68e76502c588-kube-api-access-mfsvx\") pod \"payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.185609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.185479    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6021844b-979f-42bf-b291-e21a7568aad8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.186103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.186082    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/786a0536-42fe-430f-a65e-68e76502c588-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.186148 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.186112    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6021844b-979f-42bf-b291-e21a7568aad8-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.194419 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.194394    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfsvx\" (UniqueName: \"kubernetes.io/projected/786a0536-42fe-430f-a65e-68e76502c588-kube-api-access-mfsvx\") pod \"payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.194705 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.194687    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hb9lb\" (UniqueName: \"kubernetes.io/projected/6021844b-979f-42bf-b291-e21a7568aad8-kube-api-access-hb9lb\") pod \"payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.235029 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.234999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.261967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.261931    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.488938 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.488902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:37.516510 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.516483    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"]
Sep 19 12:44:37.518800 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:44:37.518771    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786a0536_42fe_430f_a65e_68e76502c588.slice/crio-59395a50aab6e5af58e971194ebba5c1f6dbb0a4dfa0a8d0f74ece270a053ff9 WatchSource:0}: Error finding container 59395a50aab6e5af58e971194ebba5c1f6dbb0a4dfa0a8d0f74ece270a053ff9: Status 404 returned error can't find the container with id 59395a50aab6e5af58e971194ebba5c1f6dbb0a4dfa0a8d0f74ece270a053ff9
Sep 19 12:44:37.554512 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.554491    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"]
Sep 19 12:44:37.556402 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:44:37.556374    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6021844b_979f_42bf_b291_e21a7568aad8.slice/crio-ddd92c22d0411d72eeee5a02126fc976b9779853105c4b8d3eb5e26362938a16 WatchSource:0}: Error finding container ddd92c22d0411d72eeee5a02126fc976b9779853105c4b8d3eb5e26362938a16: Status 404 returned error can't find the container with id ddd92c22d0411d72eeee5a02126fc976b9779853105c4b8d3eb5e26362938a16
Sep 19 12:44:37.587874 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.587842    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:37.696258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.696224    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:37.889556 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.889444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756" event={"ID":"786a0536-42fe-430f-a65e-68e76502c588","Type":"ContainerStarted","Data":"ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57"}
Sep 19 12:44:37.889556 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.889518    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756" event={"ID":"786a0536-42fe-430f-a65e-68e76502c588","Type":"ContainerStarted","Data":"59395a50aab6e5af58e971194ebba5c1f6dbb0a4dfa0a8d0f74ece270a053ff9"}
Sep 19 12:44:37.889765 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.889649    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:37.891190 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.891163    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c" event={"ID":"6021844b-979f-42bf-b291-e21a7568aad8","Type":"ContainerStarted","Data":"32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7"}
Sep 19 12:44:37.891337 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.891212    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c" event={"ID":"6021844b-979f-42bf-b291-e21a7568aad8","Type":"ContainerStarted","Data":"ddd92c22d0411d72eeee5a02126fc976b9779853105c4b8d3eb5e26362938a16"}
Sep 19 12:44:37.891337 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.891240    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:37.906180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.906133    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756" podStartSLOduration=1.906119902 podStartE2EDuration="1.906119902s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:44:37.904967339 +0000 UTC m=+1161.265210281" watchObservedRunningTime="2026-09-19 12:44:37.906119902 +0000 UTC m=+1161.266362844"
Sep 19 12:44:37.922187 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:37.922142    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c" podStartSLOduration=1.9221293560000001 podStartE2EDuration="1.922129356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:44:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:44:37.921352899 +0000 UTC m=+1161.281595843" watchObservedRunningTime="2026-09-19 12:44:37.922129356 +0000 UTC m=+1161.282372296"
Sep 19 12:44:48.898449 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:48.898419    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:44:48.898913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:48.898504    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:44:56.572154 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:56.572125    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:59.054168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:59.054124    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:44:59.872020 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:44:59.871975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:00.143203 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.143124    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"]
Sep 19 12:45:00.146726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.146702    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"
Sep 19 12:45:00.151019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.151000    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:45:00.160713 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.160657    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"
Sep 19 12:45:00.162611 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.162592    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"]
Sep 19 12:45:00.165432 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.165397    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"]
Sep 19 12:45:00.169344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.169323    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"
Sep 19 12:45:00.179344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.179281    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"]
Sep 19 12:45:00.200792 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.200771    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"
Sep 19 12:45:00.441705 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.441682    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"]
Sep 19 12:45:00.443491 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:45:00.443445    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c8f82ea_de49_46c0_b200_6334ef6fb188.slice/crio-c0f3fa20ca9d7228e623921f1792d18573cdb77e151860407dce4792b58d4ae5 WatchSource:0}: Error finding container c0f3fa20ca9d7228e623921f1792d18573cdb77e151860407dce4792b58d4ae5: Status 404 returned error can't find the container with id c0f3fa20ca9d7228e623921f1792d18573cdb77e151860407dce4792b58d4ae5
Sep 19 12:45:00.476557 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.476529    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"]
Sep 19 12:45:00.477821 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:45:00.477788    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e0b2af_545c_4d22_9b66_0812d1d633ff.slice/crio-9b7897f3787849d4f4834f7ca9d5df7d0d2d5092c60d048fd24ca7b7b8db9b33 WatchSource:0}: Error finding container 9b7897f3787849d4f4834f7ca9d5df7d0d2d5092c60d048fd24ca7b7b8db9b33: Status 404 returned error can't find the container with id 9b7897f3787849d4f4834f7ca9d5df7d0d2d5092c60d048fd24ca7b7b8db9b33
Sep 19 12:45:00.597440 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:00.597388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:01.004684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:01.004654    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w" event={"ID":"d9e0b2af-545c-4d22-9b66-0812d1d633ff","Type":"ContainerStarted","Data":"9b7897f3787849d4f4834f7ca9d5df7d0d2d5092c60d048fd24ca7b7b8db9b33"}
Sep 19 12:45:01.005854 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:01.005832    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524" event={"ID":"3c8f82ea-de49-46c0-b200-6334ef6fb188","Type":"ContainerStarted","Data":"c0f3fa20ca9d7228e623921f1792d18573cdb77e151860407dce4792b58d4ae5"}
Sep 19 12:45:01.275452 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:01.275354    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:02.032418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:02.032382    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:02.135127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:02.135091    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:02.440536 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:02.440450    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:45:02.751109 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:02.751071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:03.566750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:03.566711    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:04.024296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:04.024266    2571 generic.go:356] "Generic (PLEG): container finished" podID="d9e0b2af-545c-4d22-9b66-0812d1d633ff" containerID="9b29bc48c768158a9498ade9d1c9b3c496ba6b1275e2f8b0a94964e9222e8638" exitCode=0
Sep 19 12:45:04.024499 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:04.024327    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w" event={"ID":"d9e0b2af-545c-4d22-9b66-0812d1d633ff","Type":"ContainerDied","Data":"9b29bc48c768158a9498ade9d1c9b3c496ba6b1275e2f8b0a94964e9222e8638"}
Sep 19 12:45:05.163707 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:05.163683    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"
Sep 19 12:45:06.035305 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:06.035276    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"
Sep 19 12:45:06.035503 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:06.035324    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w" event={"ID":"d9e0b2af-545c-4d22-9b66-0812d1d633ff","Type":"ContainerDied","Data":"9b7897f3787849d4f4834f7ca9d5df7d0d2d5092c60d048fd24ca7b7b8db9b33"}
Sep 19 12:45:06.035503 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:06.035366    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b7897f3787849d4f4834f7ca9d5df7d0d2d5092c60d048fd24ca7b7b8db9b33"
Sep 19 12:45:09.050602 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:09.050574    2571 generic.go:356] "Generic (PLEG): container finished" podID="3c8f82ea-de49-46c0-b200-6334ef6fb188" containerID="0bb4126db37de3fc31904e627c73b7a777aecf495a389bfd736eda12754a2319" exitCode=0
Sep 19 12:45:09.050953 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:09.050653    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524" event={"ID":"3c8f82ea-de49-46c0-b200-6334ef6fb188","Type":"ContainerDied","Data":"0bb4126db37de3fc31904e627c73b7a777aecf495a389bfd736eda12754a2319"}
Sep 19 12:45:10.184867 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:10.184846    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"
Sep 19 12:45:11.061737 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:11.061647    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"
Sep 19 12:45:11.061737 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:11.061675    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524" event={"ID":"3c8f82ea-de49-46c0-b200-6334ef6fb188","Type":"ContainerDied","Data":"c0f3fa20ca9d7228e623921f1792d18573cdb77e151860407dce4792b58d4ae5"}
Sep 19 12:45:11.061737 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:11.061725    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c0f3fa20ca9d7228e623921f1792d18573cdb77e151860407dce4792b58d4ae5"
Sep 19 12:45:11.406880 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:11.406793    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:11.721211 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:11.721174    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:12.049997 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:12.049909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:13.666959 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:13.666916    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:14.337292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.337254    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:14.448645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.448603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:14.547684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.547654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:14.637412 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637335    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"]
Sep 19 12:45:14.637700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637687    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c8f82ea-de49-46c0-b200-6334ef6fb188" containerName="cleanup"
Sep 19 12:45:14.637755 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637701    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c8f82ea-de49-46c0-b200-6334ef6fb188" containerName="cleanup"
Sep 19 12:45:14.637755 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637715    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e0b2af-545c-4d22-9b66-0812d1d633ff" containerName="cleanup"
Sep 19 12:45:14.637755 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637721    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e0b2af-545c-4d22-9b66-0812d1d633ff" containerName="cleanup"
Sep 19 12:45:14.637847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637778    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e0b2af-545c-4d22-9b66-0812d1d633ff" containerName="cleanup"
Sep 19 12:45:14.637847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.637788    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c8f82ea-de49-46c0-b200-6334ef6fb188" containerName="cleanup"
Sep 19 12:45:14.640874 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.640859    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.643640 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.643621    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"distinct-w0-a2e924-kserve-self-signed-certs\""
Sep 19 12:45:14.649453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.649432    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"]
Sep 19 12:45:14.731827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.731798    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-home\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.731827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.731831    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-model-cache\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.732251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.731862    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-dshm\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.732251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.731924    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-tmp-dir\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.732251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.731969    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7533bbc7-efa7-438f-843f-9ea8142e8701-tls-certs\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.732251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.731993    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mwht\" (UniqueName: \"kubernetes.io/projected/7533bbc7-efa7-438f-843f-9ea8142e8701-kube-api-access-8mwht\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.743327 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.743300    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:14.832758 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.832717    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7533bbc7-efa7-438f-843f-9ea8142e8701-tls-certs\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.832932 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.832811    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mwht\" (UniqueName: \"kubernetes.io/projected/7533bbc7-efa7-438f-843f-9ea8142e8701-kube-api-access-8mwht\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.832932 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.832899    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-home\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.833177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.833154    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-model-cache\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.833284 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.833222    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-dshm\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.833351 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.833305    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-tmp-dir\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.833400 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.833358    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-home\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.833500 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.833454    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-model-cache\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.833641 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.833626    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-tmp-dir\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.835234 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.835213    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-dshm\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.835368 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.835350    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7533bbc7-efa7-438f-843f-9ea8142e8701-tls-certs\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.843588 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.843565    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mwht\" (UniqueName: \"kubernetes.io/projected/7533bbc7-efa7-438f-843f-9ea8142e8701-kube-api-access-8mwht\") pod \"distinct-w0-a2e924-kserve-ddd995fbd-mkg92\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:14.950888 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:14.950862    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:15.225605 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:15.225579    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"]
Sep 19 12:45:15.226922 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:45:15.226892    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7533bbc7_efa7_438f_843f_9ea8142e8701.slice/crio-3f7273215cca58a274f26f489b5e514b4650441a350487d3e5368c82f2fb0876 WatchSource:0}: Error finding container 3f7273215cca58a274f26f489b5e514b4650441a350487d3e5368c82f2fb0876: Status 404 returned error can't find the container with id 3f7273215cca58a274f26f489b5e514b4650441a350487d3e5368c82f2fb0876
Sep 19 12:45:15.498033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:15.497947    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:16.089177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:16.089140    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92" event={"ID":"7533bbc7-efa7-438f-843f-9ea8142e8701","Type":"ContainerStarted","Data":"9ee46549352c15d10503132f71405b0e54436d361d0bea97b4ed0f7cafe3bd89"}
Sep 19 12:45:16.089608 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:16.089188    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92" event={"ID":"7533bbc7-efa7-438f-843f-9ea8142e8701","Type":"ContainerStarted","Data":"3f7273215cca58a274f26f489b5e514b4650441a350487d3e5368c82f2fb0876"}
Sep 19 12:45:16.109374 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:16.109327    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92" podStartSLOduration=2.109313547 podStartE2EDuration="2.109313547s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:16.109278997 +0000 UTC m=+1199.469521939" watchObservedRunningTime="2026-09-19 12:45:16.109313547 +0000 UTC m=+1199.469556484"
Sep 19 12:45:16.383837 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:16.383751    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:17.352754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:17.352714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:18.223814 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:18.223773    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:19.112262 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:19.112225    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:19.889155 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:19.889102    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:19.993757 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:19.993715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:20.495547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:20.495519    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:21.848296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:21.848255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:24.952054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:24.952018    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:24.952409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:24.952071    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:24.963987 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:24.963962    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:25.144547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:25.144521    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:45:28.454519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:28.454485    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:45:30.282441 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:30.282407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:31.562338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:31.562276    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:41.556888 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:41.556855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:43.069122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:43.069073    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:43.962011 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:43.961977    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:44.331485 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:44.331371    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:44.887052 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:44.887020    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:46.001249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:46.001212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:46.884939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:46.884876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:47.809045 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:47.809006    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:48.851232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:48.851197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:49.678730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:49.678698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:58.354001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.353973    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"]
Sep 19 12:45:58.357391 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.357373    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.361539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.361514    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-e8f833\"/\"kube-root-ca.crt\""
Sep 19 12:45:58.361662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.361540    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-e8f833\"/\"default-dockercfg-mwj6s\""
Sep 19 12:45:58.361662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.361552    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-e8f833\"/\"noten-model-e8f833-kserve-self-signed-certs\""
Sep 19 12:45:58.361662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.361596    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-e8f833\"/\"openshift-service-ca.crt\""
Sep 19 12:45:58.370367 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.370346    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"]
Sep 19 12:45:58.388169 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.388144    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:45:58.421147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.421124    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-home\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.421233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.421173    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-tmp-dir\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.421233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.421221    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dmt7\" (UniqueName: \"kubernetes.io/projected/0c63c404-d085-4799-8b30-f20fa39ce3fb-kube-api-access-8dmt7\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.421304 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.421282    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-dshm\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.421381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.421300    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c63c404-d085-4799-8b30-f20fa39ce3fb-tls-certs\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.421381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.421326    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-model-cache\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522438    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-tmp-dir\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522489    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dmt7\" (UniqueName: \"kubernetes.io/projected/0c63c404-d085-4799-8b30-f20fa39ce3fb-kube-api-access-8dmt7\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522543    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-dshm\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522559    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c63c404-d085-4799-8b30-f20fa39ce3fb-tls-certs\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522584    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-model-cache\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522607 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522603    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-home\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522852    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-tmp-dir\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.522960 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.522942    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-model-cache\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.523017 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.523003    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-home\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.524747 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.524729    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-dshm\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.524963 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.524946    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c63c404-d085-4799-8b30-f20fa39ce3fb-tls-certs\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.532422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.532401    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dmt7\" (UniqueName: \"kubernetes.io/projected/0c63c404-d085-4799-8b30-f20fa39ce3fb-kube-api-access-8dmt7\") pod \"noten-model-e8f833-kserve-77b996b5b7-kk747\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") " pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.667057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.667003    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:45:58.944196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:58.944171    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"]
Sep 19 12:45:58.946225 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:45:58.946197    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c63c404_d085_4799_8b30_f20fa39ce3fb.slice/crio-65574cbde242f81e0c33888fd89d73f439b7216d2a122c5ce79fbefd52caed83 WatchSource:0}: Error finding container 65574cbde242f81e0c33888fd89d73f439b7216d2a122c5ce79fbefd52caed83: Status 404 returned error can't find the container with id 65574cbde242f81e0c33888fd89d73f439b7216d2a122c5ce79fbefd52caed83
Sep 19 12:45:59.304588 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:59.304556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747" event={"ID":"0c63c404-d085-4799-8b30-f20fa39ce3fb","Type":"ContainerStarted","Data":"58c8a0e064be031026057a332d1064301df6885b309da35c6b871de432e18d47"}
Sep 19 12:45:59.304843 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:59.304596    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747" event={"ID":"0c63c404-d085-4799-8b30-f20fa39ce3fb","Type":"ContainerStarted","Data":"65574cbde242f81e0c33888fd89d73f439b7216d2a122c5ce79fbefd52caed83"}
Sep 19 12:45:59.334891 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:45:59.334849    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747" podStartSLOduration=1.334836062 podStartE2EDuration="1.334836062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:45:59.333081391 +0000 UTC m=+1242.693324332" watchObservedRunningTime="2026-09-19 12:45:59.334836062 +0000 UTC m=+1242.695079003"
Sep 19 12:46:02.320536 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:02.320445    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"]
Sep 19 12:46:02.347606 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:02.347576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:07.973075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:07.973039    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"]
Sep 19 12:46:08.009509 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:08.009451    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:08.426284 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:08.426198    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:08.667716 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:08.667669    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:46:08.667894 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:08.667752    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:46:08.680243 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:08.680166    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:46:09.188583 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:09.188552    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:09.371124 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:09.371099    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:46:10.050577 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:10.050542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:10.808092 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:10.808055    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:11.470653 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:11.470606    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:11.540385 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:11.540354    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:46:12.276117 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:12.276070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:12.972145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:12.972107    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:13.750814 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:13.750778    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:14.586279 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:14.586237    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:17.284737 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.284707    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"]
Sep 19 12:46:17.288108 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.288092    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.291442 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.291417    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9a75910\"/\"default-dockercfg-jx2sq\""
Sep 19 12:46:17.292600 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.292584    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9a75910\"/\"openshift-service-ca.crt\""
Sep 19 12:46:17.292703 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.292606    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e9a75910\"/\"kube-root-ca.crt\""
Sep 19 12:46:17.292703 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.292587    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e9a75910\"/\"e2e-lifecycle-model-e9a75910-kserve-self-signed-certs\""
Sep 19 12:46:17.303154 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.303129    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"]
Sep 19 12:46:17.343653 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.343612    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl" podUID="009e7b72-924d-4291-8acc-5a2b3a7bb0e5" containerName="main" containerID="cri-o://116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b" gracePeriod=45
Sep 19 12:46:17.355916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.355890    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:17.398246 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.398218    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de678c4e-4602-4079-b49e-05f9b26d8f7f-tls-certs\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.398404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.398270    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjzpl\" (UniqueName: \"kubernetes.io/projected/de678c4e-4602-4079-b49e-05f9b26d8f7f-kube-api-access-zjzpl\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.398404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.398333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-dshm\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.398404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.398361    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-home\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.398404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.398391    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-tmp-dir\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.398558 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.398443    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-model-cache\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.499900 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.499859    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de678c4e-4602-4079-b49e-05f9b26d8f7f-tls-certs\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.499909    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjzpl\" (UniqueName: \"kubernetes.io/projected/de678c4e-4602-4079-b49e-05f9b26d8f7f-kube-api-access-zjzpl\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.499934    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-dshm\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.499960    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-home\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.499974    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-tmp-dir\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.500028    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-model-cache\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500515 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.500483    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-model-cache\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500658 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.500523    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-home\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.500658 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.500559    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-tmp-dir\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.502215 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.502193    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-dshm\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.502877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.502853    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de678c4e-4602-4079-b49e-05f9b26d8f7f-tls-certs\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.515085 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.515057    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjzpl\" (UniqueName: \"kubernetes.io/projected/de678c4e-4602-4079-b49e-05f9b26d8f7f-kube-api-access-zjzpl\") pod \"e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") " pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.586817 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.586795    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:46:17.598043 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.598019    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:17.701363 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701339    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-model-cache\") pod \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") "
Sep 19 12:46:17.701539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701393    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq4nf\" (UniqueName: \"kubernetes.io/projected/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-kube-api-access-nq4nf\") pod \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") "
Sep 19 12:46:17.701539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701433    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-home\") pod \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") "
Sep 19 12:46:17.701539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701517    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tmp-dir\") pod \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") "
Sep 19 12:46:17.701724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701576    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-dshm\") pod \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") "
Sep 19 12:46:17.701724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701607    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tls-certs\") pod \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\" (UID: \"009e7b72-924d-4291-8acc-5a2b3a7bb0e5\") "
Sep 19 12:46:17.701724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701691    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-model-cache" (OuterVolumeSpecName: "model-cache") pod "009e7b72-924d-4291-8acc-5a2b3a7bb0e5" (UID: "009e7b72-924d-4291-8acc-5a2b3a7bb0e5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:17.701866 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701829    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "009e7b72-924d-4291-8acc-5a2b3a7bb0e5" (UID: "009e7b72-924d-4291-8acc-5a2b3a7bb0e5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:17.701866 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.701843    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-home" (OuterVolumeSpecName: "home") pod "009e7b72-924d-4291-8acc-5a2b3a7bb0e5" (UID: "009e7b72-924d-4291-8acc-5a2b3a7bb0e5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:17.702032 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.702009    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:17.702032 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.702032    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:17.702202 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.702043    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:17.703839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.703816    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-dshm" (OuterVolumeSpecName: "dshm") pod "009e7b72-924d-4291-8acc-5a2b3a7bb0e5" (UID: "009e7b72-924d-4291-8acc-5a2b3a7bb0e5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:17.703903 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.703867    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "009e7b72-924d-4291-8acc-5a2b3a7bb0e5" (UID: "009e7b72-924d-4291-8acc-5a2b3a7bb0e5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:46:17.703903 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.703879    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-kube-api-access-nq4nf" (OuterVolumeSpecName: "kube-api-access-nq4nf") pod "009e7b72-924d-4291-8acc-5a2b3a7bb0e5" (UID: "009e7b72-924d-4291-8acc-5a2b3a7bb0e5"). InnerVolumeSpecName "kube-api-access-nq4nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:17.802574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.802549    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:17.802574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.802570    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:17.802574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.802582    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq4nf\" (UniqueName: \"kubernetes.io/projected/009e7b72-924d-4291-8acc-5a2b3a7bb0e5-kube-api-access-nq4nf\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:17.869351 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:17.869327    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"]
Sep 19 12:46:17.871440 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:46:17.871417    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde678c4e_4602_4079_b49e_05f9b26d8f7f.slice/crio-d394244e794b55df7efeba696540ac4e621e6616378940d2741d3e62c5342aca WatchSource:0}: Error finding container d394244e794b55df7efeba696540ac4e621e6616378940d2741d3e62c5342aca: Status 404 returned error can't find the container with id d394244e794b55df7efeba696540ac4e621e6616378940d2741d3e62c5342aca
Sep 19 12:46:18.407168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.407132    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h" event={"ID":"de678c4e-4602-4079-b49e-05f9b26d8f7f","Type":"ContainerStarted","Data":"79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768"}
Sep 19 12:46:18.407612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.407179    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h" event={"ID":"de678c4e-4602-4079-b49e-05f9b26d8f7f","Type":"ContainerStarted","Data":"d394244e794b55df7efeba696540ac4e621e6616378940d2741d3e62c5342aca"}
Sep 19 12:46:18.408552 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.408528    2571 generic.go:356] "Generic (PLEG): container finished" podID="009e7b72-924d-4291-8acc-5a2b3a7bb0e5" containerID="116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b" exitCode=0
Sep 19 12:46:18.408664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.408574    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"
Sep 19 12:46:18.408664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.408607    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl" event={"ID":"009e7b72-924d-4291-8acc-5a2b3a7bb0e5","Type":"ContainerDied","Data":"116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b"}
Sep 19 12:46:18.408664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.408645    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl" event={"ID":"009e7b72-924d-4291-8acc-5a2b3a7bb0e5","Type":"ContainerDied","Data":"e538016f259055573adcf2afdd195cea8a7bb6df96c510e78bce485403628044"}
Sep 19 12:46:18.408768 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.408666    2571 scope.go:117] "RemoveContainer" containerID="116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b"
Sep 19 12:46:18.419160 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.419137    2571 scope.go:117] "RemoveContainer" containerID="116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b"
Sep 19 12:46:18.419747 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:46:18.419710    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b\": container with ID starting with 116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b not found: ID does not exist" containerID="116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b"
Sep 19 12:46:18.419838 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.419755    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b"} err="failed to get container status \"116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b\": rpc error: code = NotFound desc = could not find container \"116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b\": container with ID starting with 116b3272f22c8658942d7150478c5e3a7ea8d6c989aacfc0d5ed556252cbaa8b not found: ID does not exist"
Sep 19 12:46:18.426278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.426234    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h" podStartSLOduration=1.426221053 podStartE2EDuration="1.426221053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:46:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:46:18.425211806 +0000 UTC m=+1261.785454748" watchObservedRunningTime="2026-09-19 12:46:18.426221053 +0000 UTC m=+1261.786463993"
Sep 19 12:46:18.441694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.441669    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"]
Sep 19 12:46:18.446957 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.446935    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-61583b/test-model-61583b-kserve-9fb6f86c5-pkjnl"]
Sep 19 12:46:18.672766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.672680    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:18.740599 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.740567    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="009e7b72-924d-4291-8acc-5a2b3a7bb0e5" path="/var/lib/kubelet/pods/009e7b72-924d-4291-8acc-5a2b3a7bb0e5/volumes"
Sep 19 12:46:18.743976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.743948    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:18.811175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.811127    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:18.887260 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.887225    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:18.963060 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:18.963024    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:19.281420 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:19.281321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:19.401669 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:19.401631    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:20.556702 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:20.556666    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:21.673200 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:21.673162    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:22.794366 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:22.794327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:22.992180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:22.992140    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv" podUID="e69ebde9-1fe9-461b-9b7c-9110d34388a8" containerName="main" containerID="cri-o://d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d" gracePeriod=45
Sep 19 12:46:23.251850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.251824    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:46:23.359594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359514    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhxh6\" (UniqueName: \"kubernetes.io/projected/e69ebde9-1fe9-461b-9b7c-9110d34388a8-kube-api-access-vhxh6\") pod \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") "
Sep 19 12:46:23.359594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359551    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-dshm\") pod \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") "
Sep 19 12:46:23.359822 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359610    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tls-certs\") pod \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") "
Sep 19 12:46:23.359822 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359641    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tmp-dir\") pod \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") "
Sep 19 12:46:23.359822 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359676    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-model-cache\") pod \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") "
Sep 19 12:46:23.359822 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359773    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-home\") pod \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\" (UID: \"e69ebde9-1fe9-461b-9b7c-9110d34388a8\") "
Sep 19 12:46:23.360025 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359945    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e69ebde9-1fe9-461b-9b7c-9110d34388a8" (UID: "e69ebde9-1fe9-461b-9b7c-9110d34388a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:23.360025 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.359987    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "e69ebde9-1fe9-461b-9b7c-9110d34388a8" (UID: "e69ebde9-1fe9-461b-9b7c-9110d34388a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:23.360177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.360158    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:23.360237 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.360213    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:23.360295 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.360233    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-home" (OuterVolumeSpecName: "home") pod "e69ebde9-1fe9-461b-9b7c-9110d34388a8" (UID: "e69ebde9-1fe9-461b-9b7c-9110d34388a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:23.361942 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.361916    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-dshm" (OuterVolumeSpecName: "dshm") pod "e69ebde9-1fe9-461b-9b7c-9110d34388a8" (UID: "e69ebde9-1fe9-461b-9b7c-9110d34388a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:23.362053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.361969    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e69ebde9-1fe9-461b-9b7c-9110d34388a8" (UID: "e69ebde9-1fe9-461b-9b7c-9110d34388a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:46:23.362053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.362005    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e69ebde9-1fe9-461b-9b7c-9110d34388a8-kube-api-access-vhxh6" (OuterVolumeSpecName: "kube-api-access-vhxh6") pod "e69ebde9-1fe9-461b-9b7c-9110d34388a8" (UID: "e69ebde9-1fe9-461b-9b7c-9110d34388a8"). InnerVolumeSpecName "kube-api-access-vhxh6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:23.434799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.434762    2571 generic.go:356] "Generic (PLEG): container finished" podID="e69ebde9-1fe9-461b-9b7c-9110d34388a8" containerID="d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d" exitCode=0
Sep 19 12:46:23.434989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.434826    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"
Sep 19 12:46:23.434989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.434855    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv" event={"ID":"e69ebde9-1fe9-461b-9b7c-9110d34388a8","Type":"ContainerDied","Data":"d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d"}
Sep 19 12:46:23.434989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.434900    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv" event={"ID":"e69ebde9-1fe9-461b-9b7c-9110d34388a8","Type":"ContainerDied","Data":"b3982c282c71f521fab48e93c802cc4d8d267c012e2ff28c3473ea30a827e6f2"}
Sep 19 12:46:23.434989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.434917    2571 scope.go:117] "RemoveContainer" containerID="d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d"
Sep 19 12:46:23.444090 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.444069    2571 scope.go:117] "RemoveContainer" containerID="d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d"
Sep 19 12:46:23.444370 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:46:23.444353    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d\": container with ID starting with d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d not found: ID does not exist" containerID="d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d"
Sep 19 12:46:23.444438 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.444377    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d"} err="failed to get container status \"d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d\": rpc error: code = NotFound desc = could not find container \"d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d\": container with ID starting with d0fa77aecaf6118a8739e65f3948474b8f2233d9d22d6f7fd3f96cf389819b7d not found: ID does not exist"
Sep 19 12:46:23.461398 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.461370    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:23.461398 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.461396    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhxh6\" (UniqueName: \"kubernetes.io/projected/e69ebde9-1fe9-461b-9b7c-9110d34388a8-kube-api-access-vhxh6\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:23.461545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.461411    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e69ebde9-1fe9-461b-9b7c-9110d34388a8-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:23.461545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.461424    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e69ebde9-1fe9-461b-9b7c-9110d34388a8-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:23.470224 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.470202    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"]
Sep 19 12:46:23.474692 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.474671    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-5be349/test-model-5be349-kserve-7b5cb4fd5c-tr8sv"]
Sep 19 12:46:23.806964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:23.806929    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:24.740233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:24.740200    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e69ebde9-1fe9-461b-9b7c-9110d34388a8" path="/var/lib/kubelet/pods/e69ebde9-1fe9-461b-9b7c-9110d34388a8/volumes"
Sep 19 12:46:24.916384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:24.916343    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:26.155594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:26.155560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:27.154962 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.154925    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:27.259162 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.259132    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:27.471137 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.471100    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:27.598637 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.598604    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:27.598820 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.598657    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:27.611010 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.610988    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:27.955633 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:27.955598    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:28.473342 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:28.473318    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:46:28.801828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:28.801740    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:29.662239 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:29.662199    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:30.426943 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:30.426912    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:31.033379 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:31.033333    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:31.261847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:31.261813    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:31.361826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:31.361733    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:31.461371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:31.461340    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:32.108679 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:32.108639    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:32.907049 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:32.907008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:33.652892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:33.652855    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:36.376880 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:36.376844    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"]
Sep 19 12:46:36.424333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:36.424303    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:38.262993 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.262937    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"]
Sep 19 12:46:38.263483 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.263169    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r" podUID="951394ab-552a-4fc9-a144-a6f4b0606896" containerName="maas-api" containerID="cri-o://87b5aaaee728ba70d589a0982457cda13a50e3259681bfb895c09d0fe45f2a4e" gracePeriod=30
Sep 19 12:46:38.292899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.292859    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:38.369040 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.369006    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:38.436755 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.436720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:38.513739 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.513664    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:38.515350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.515324    2571 generic.go:356] "Generic (PLEG): container finished" podID="951394ab-552a-4fc9-a144-a6f4b0606896" containerID="87b5aaaee728ba70d589a0982457cda13a50e3259681bfb895c09d0fe45f2a4e" exitCode=0
Sep 19 12:46:38.515497 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.515396    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r" event={"ID":"951394ab-552a-4fc9-a144-a6f4b0606896","Type":"ContainerDied","Data":"87b5aaaee728ba70d589a0982457cda13a50e3259681bfb895c09d0fe45f2a4e"}
Sep 19 12:46:38.515497 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.515452    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r" event={"ID":"951394ab-552a-4fc9-a144-a6f4b0606896","Type":"ContainerDied","Data":"45d40b36d455ff2da57d1279ea4432cce03995d493d4e529586703f755e929a4"}
Sep 19 12:46:38.515497 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.515493    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="45d40b36d455ff2da57d1279ea4432cce03995d493d4e529586703f755e929a4"
Sep 19 12:46:38.516250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.516234    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:46:38.607373 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.607341    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-maas-api-tls\") pod \"951394ab-552a-4fc9-a144-a6f4b0606896\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") "
Sep 19 12:46:38.607587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.607398    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-metrics-certs\") pod \"951394ab-552a-4fc9-a144-a6f4b0606896\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") "
Sep 19 12:46:38.607587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.607510    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951394ab-552a-4fc9-a144-a6f4b0606896-openshift-service-ca\") pod \"951394ab-552a-4fc9-a144-a6f4b0606896\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") "
Sep 19 12:46:38.607587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.607539    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m9nf6\" (UniqueName: \"kubernetes.io/projected/951394ab-552a-4fc9-a144-a6f4b0606896-kube-api-access-m9nf6\") pod \"951394ab-552a-4fc9-a144-a6f4b0606896\" (UID: \"951394ab-552a-4fc9-a144-a6f4b0606896\") "
Sep 19 12:46:38.607901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.607875    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/951394ab-552a-4fc9-a144-a6f4b0606896-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "951394ab-552a-4fc9-a144-a6f4b0606896" (UID: "951394ab-552a-4fc9-a144-a6f4b0606896"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:46:38.609530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.609503    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "951394ab-552a-4fc9-a144-a6f4b0606896" (UID: "951394ab-552a-4fc9-a144-a6f4b0606896"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:46:38.609620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.609564    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/951394ab-552a-4fc9-a144-a6f4b0606896-kube-api-access-m9nf6" (OuterVolumeSpecName: "kube-api-access-m9nf6") pod "951394ab-552a-4fc9-a144-a6f4b0606896" (UID: "951394ab-552a-4fc9-a144-a6f4b0606896"). InnerVolumeSpecName "kube-api-access-m9nf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:38.609620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.609599    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "951394ab-552a-4fc9-a144-a6f4b0606896" (UID: "951394ab-552a-4fc9-a144-a6f4b0606896"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:46:38.709010 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.708980    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/951394ab-552a-4fc9-a144-a6f4b0606896-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.709010 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.709004    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9nf6\" (UniqueName: \"kubernetes.io/projected/951394ab-552a-4fc9-a144-a6f4b0606896-kube-api-access-m9nf6\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.709010 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.709014    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.709246 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.709023    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/951394ab-552a-4fc9-a144-a6f4b0606896-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:38.853786 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.853692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:38.983304 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:38.983247    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:39.519789 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:39.519758    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"
Sep 19 12:46:39.540028 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:39.539996    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"]
Sep 19 12:46:39.544158 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:39.544136    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5be349-7c7bbd78c-z8g5r"]
Sep 19 12:46:39.701394 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:39.701350    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:40.740237 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:40.740203    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="951394ab-552a-4fc9-a144-a6f4b0606896" path="/var/lib/kubelet/pods/951394ab-552a-4fc9-a144-a6f4b0606896/volumes"
Sep 19 12:46:40.760204 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:40.760167    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:41.508896 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:41.508864    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:42.158000 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:42.157928    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:42.901361 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:42.901327    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:44.631598 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:44.631564    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:46:45.146405 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:45.146375    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:45.243828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:45.243798    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:45.391193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:45.391162    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:45.925894 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:45.925865    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:46.238186 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:46.238157    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:46.494500 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:46.494372    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:48.743315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:48.743279    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:48.902655 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:48.902621    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:49.025917 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.025826    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:49.199431 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199401    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"]
Sep 19 12:46:49.199828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199810    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="009e7b72-924d-4291-8acc-5a2b3a7bb0e5" containerName="main"
Sep 19 12:46:49.199908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199831    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="009e7b72-924d-4291-8acc-5a2b3a7bb0e5" containerName="main"
Sep 19 12:46:49.199908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199869    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="951394ab-552a-4fc9-a144-a6f4b0606896" containerName="maas-api"
Sep 19 12:46:49.199908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199878    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="951394ab-552a-4fc9-a144-a6f4b0606896" containerName="maas-api"
Sep 19 12:46:49.199908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199889    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e69ebde9-1fe9-461b-9b7c-9110d34388a8" containerName="main"
Sep 19 12:46:49.199908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199897    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e69ebde9-1fe9-461b-9b7c-9110d34388a8" containerName="main"
Sep 19 12:46:49.200161 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.199994    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="951394ab-552a-4fc9-a144-a6f4b0606896" containerName="maas-api"
Sep 19 12:46:49.200161 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.200006    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="009e7b72-924d-4291-8acc-5a2b3a7bb0e5" containerName="main"
Sep 19 12:46:49.200161 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.200016    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e69ebde9-1fe9-461b-9b7c-9110d34388a8" containerName="main"
Sep 19 12:46:49.204438 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.204418    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.207729 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.207708    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"e2e-conflict-b-3ed226d2-kserve-self-signed-certs\""
Sep 19 12:46:49.218181 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.218159    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"]
Sep 19 12:46:49.256989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.256940    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:49.315225 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.315163    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-model-cache\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.315225 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.315196    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tls-certs\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.315370 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.315289    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-dshm\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.315370 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.315317    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-home\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.315370 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.315343    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tmp-dir\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.315481 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.315394    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4hd9\" (UniqueName: \"kubernetes.io/projected/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-kube-api-access-h4hd9\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.416911 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.416879    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-dshm\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.416911 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.416916    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-home\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.416937    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tmp-dir\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.416980    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4hd9\" (UniqueName: \"kubernetes.io/projected/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-kube-api-access-h4hd9\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.417002    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-model-cache\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.417023    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tls-certs\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417391 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.417364    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-home\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.417398    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tmp-dir\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.417479 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.417438    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-model-cache\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.419243 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.419221    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-dshm\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.419421 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.419406    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tls-certs\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.428827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.428808    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4hd9\" (UniqueName: \"kubernetes.io/projected/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-kube-api-access-h4hd9\") pod \"e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") " pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.514855 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.514813    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:49.754907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.754876    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:49.787997 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:49.787971    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"]
Sep 19 12:46:49.789676 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:46:49.789652    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaadcd3b4_dd28_4dfb_a84a_fa3e94cc5aaa.slice/crio-9ddc250bf8b6757c739ba5d8f182e19a61bafe615cc09e20ae85dd666d35e56b WatchSource:0}: Error finding container 9ddc250bf8b6757c739ba5d8f182e19a61bafe615cc09e20ae85dd666d35e56b: Status 404 returned error can't find the container with id 9ddc250bf8b6757c739ba5d8f182e19a61bafe615cc09e20ae85dd666d35e56b
Sep 19 12:46:50.548688 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:50.548658    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:50.583549 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:50.583509    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67" event={"ID":"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa","Type":"ContainerStarted","Data":"03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73"}
Sep 19 12:46:50.583716 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:50.583556    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67" event={"ID":"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa","Type":"ContainerStarted","Data":"9ddc250bf8b6757c739ba5d8f182e19a61bafe615cc09e20ae85dd666d35e56b"}
Sep 19 12:46:50.599815 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:50.599772    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67" podStartSLOduration=1.599757058 podStartE2EDuration="1.599757058s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:46:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:46:50.599286713 +0000 UTC m=+1293.959529655" watchObservedRunningTime="2026-09-19 12:46:50.599757058 +0000 UTC m=+1293.960000035"
Sep 19 12:46:51.270638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.270602    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:51.396045 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.396005    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747" podUID="0c63c404-d085-4799-8b30-f20fa39ce3fb" containerName="main" containerID="cri-o://58c8a0e064be031026057a332d1064301df6885b309da35c6b871de432e18d47" gracePeriod=45
Sep 19 12:46:51.592136 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.592094    2571 generic.go:356] "Generic (PLEG): container finished" podID="0c63c404-d085-4799-8b30-f20fa39ce3fb" containerID="58c8a0e064be031026057a332d1064301df6885b309da35c6b871de432e18d47" exitCode=0
Sep 19 12:46:51.592332 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.592176    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747" event={"ID":"0c63c404-d085-4799-8b30-f20fa39ce3fb","Type":"ContainerDied","Data":"58c8a0e064be031026057a332d1064301df6885b309da35c6b871de432e18d47"}
Sep 19 12:46:51.648333 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.648311    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:46:51.843007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.842929    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-model-cache\") pod \"0c63c404-d085-4799-8b30-f20fa39ce3fb\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") "
Sep 19 12:46:51.843007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843003    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-home\") pod \"0c63c404-d085-4799-8b30-f20fa39ce3fb\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") "
Sep 19 12:46:51.843256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843028    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dmt7\" (UniqueName: \"kubernetes.io/projected/0c63c404-d085-4799-8b30-f20fa39ce3fb-kube-api-access-8dmt7\") pod \"0c63c404-d085-4799-8b30-f20fa39ce3fb\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") "
Sep 19 12:46:51.843256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843077    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-tmp-dir\") pod \"0c63c404-d085-4799-8b30-f20fa39ce3fb\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") "
Sep 19 12:46:51.843256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843102    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-dshm\") pod \"0c63c404-d085-4799-8b30-f20fa39ce3fb\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") "
Sep 19 12:46:51.843256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843130    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c63c404-d085-4799-8b30-f20fa39ce3fb-tls-certs\") pod \"0c63c404-d085-4799-8b30-f20fa39ce3fb\" (UID: \"0c63c404-d085-4799-8b30-f20fa39ce3fb\") "
Sep 19 12:46:51.843256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843193    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-model-cache" (OuterVolumeSpecName: "model-cache") pod "0c63c404-d085-4799-8b30-f20fa39ce3fb" (UID: "0c63c404-d085-4799-8b30-f20fa39ce3fb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:51.843530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843297    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0c63c404-d085-4799-8b30-f20fa39ce3fb" (UID: "0c63c404-d085-4799-8b30-f20fa39ce3fb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:51.843530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843305    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-home" (OuterVolumeSpecName: "home") pod "0c63c404-d085-4799-8b30-f20fa39ce3fb" (UID: "0c63c404-d085-4799-8b30-f20fa39ce3fb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:51.843613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843558    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:51.843613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843576    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:51.843613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.843584    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:51.845374 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.845351    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c63c404-d085-4799-8b30-f20fa39ce3fb-kube-api-access-8dmt7" (OuterVolumeSpecName: "kube-api-access-8dmt7") pod "0c63c404-d085-4799-8b30-f20fa39ce3fb" (UID: "0c63c404-d085-4799-8b30-f20fa39ce3fb"). InnerVolumeSpecName "kube-api-access-8dmt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:46:51.845668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.845650    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-dshm" (OuterVolumeSpecName: "dshm") pod "0c63c404-d085-4799-8b30-f20fa39ce3fb" (UID: "0c63c404-d085-4799-8b30-f20fa39ce3fb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:46:51.845720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.845658    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0c63c404-d085-4799-8b30-f20fa39ce3fb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0c63c404-d085-4799-8b30-f20fa39ce3fb" (UID: "0c63c404-d085-4799-8b30-f20fa39ce3fb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:46:51.944977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.944951    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dmt7\" (UniqueName: \"kubernetes.io/projected/0c63c404-d085-4799-8b30-f20fa39ce3fb-kube-api-access-8dmt7\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:51.944977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.944975    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0c63c404-d085-4799-8b30-f20fa39ce3fb-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:51.945123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.944984    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0c63c404-d085-4799-8b30-f20fa39ce3fb-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:46:51.953806 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:51.953775    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:52.598216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.598181    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747" event={"ID":"0c63c404-d085-4799-8b30-f20fa39ce3fb","Type":"ContainerDied","Data":"65574cbde242f81e0c33888fd89d73f439b7216d2a122c5ce79fbefd52caed83"}
Sep 19 12:46:52.598216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.598216    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"
Sep 19 12:46:52.598793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.598241    2571 scope.go:117] "RemoveContainer" containerID="58c8a0e064be031026057a332d1064301df6885b309da35c6b871de432e18d47"
Sep 19 12:46:52.621084 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.621058    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"]
Sep 19 12:46:52.625532 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.625510    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-e8f833/noten-model-e8f833-kserve-77b996b5b7-kk747"]
Sep 19 12:46:52.741597 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.741560    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c63c404-d085-4799-8b30-f20fa39ce3fb" path="/var/lib/kubelet/pods/0c63c404-d085-4799-8b30-f20fa39ce3fb/volumes"
Sep 19 12:46:52.788902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.788868    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:52.885835 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:52.885740    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:53.141250 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:53.141142    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:53.269100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:53.269070    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:53.360646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:53.360614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:53.723593 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:53.723560    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:46:59.515107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:59.515074    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:59.515525 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:59.515122    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:59.527804 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:59.527782    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:46:59.644658 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:46:59.644635    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:47:13.294439 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:13.294400    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:14.412055 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.412023    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"]
Sep 19 12:47:14.412446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.412363    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c63c404-d085-4799-8b30-f20fa39ce3fb" containerName="main"
Sep 19 12:47:14.412446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.412374    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c63c404-d085-4799-8b30-f20fa39ce3fb" containerName="main"
Sep 19 12:47:14.412446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.412442    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c63c404-d085-4799-8b30-f20fa39ce3fb" containerName="main"
Sep 19 12:47:14.415362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.415345    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.418266 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.418242    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-a2e924\"/\"distinct-2-w0-a2e924-kserve-self-signed-certs\""
Sep 19 12:47:14.425156 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.425134    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"]
Sep 19 12:47:14.439038 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.439008    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:14.541013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.540975    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-tmp-dir\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.541013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.541017    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-home\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.541288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.541037    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v2rk\" (UniqueName: \"kubernetes.io/projected/c7c386f9-b795-4dcb-a600-d38dcf5af343-kube-api-access-7v2rk\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.541288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.541204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-model-cache\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.541288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.541238    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-dshm\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.541288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.541273    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c386f9-b795-4dcb-a600-d38dcf5af343-tls-certs\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642134    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-tmp-dir\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642181    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-home\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642234    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v2rk\" (UniqueName: \"kubernetes.io/projected/c7c386f9-b795-4dcb-a600-d38dcf5af343-kube-api-access-7v2rk\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642360    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-model-cache\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642578 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642391    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-dshm\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642578 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642422    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c386f9-b795-4dcb-a600-d38dcf5af343-tls-certs\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642698 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642613    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-tmp-dir\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642698 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642680    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-home\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.642815 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.642791    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-model-cache\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.644777 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.644760    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-dshm\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.644859 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.644844    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c386f9-b795-4dcb-a600-d38dcf5af343-tls-certs\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.650380 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.650363    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v2rk\" (UniqueName: \"kubernetes.io/projected/c7c386f9-b795-4dcb-a600-d38dcf5af343-kube-api-access-7v2rk\") pod \"distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") " pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:14.725753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:14.725726    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:15.000641 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:15.000609    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"]
Sep 19 12:47:15.001848 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:47:15.001822    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7c386f9_b795_4dcb_a600_d38dcf5af343.slice/crio-3e193b37e5f4ef6ae783c5ee454116ec0067611c623d18186a64df1b99eea2e6 WatchSource:0}: Error finding container 3e193b37e5f4ef6ae783c5ee454116ec0067611c623d18186a64df1b99eea2e6: Status 404 returned error can't find the container with id 3e193b37e5f4ef6ae783c5ee454116ec0067611c623d18186a64df1b99eea2e6
Sep 19 12:47:15.716054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:15.716019    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5" event={"ID":"c7c386f9-b795-4dcb-a600-d38dcf5af343","Type":"ContainerStarted","Data":"50e52d0a2f2642bdf9e3a642c482a838d474feeea2a0cc7b72456e1f75a9044d"}
Sep 19 12:47:15.716054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:15.716061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5" event={"ID":"c7c386f9-b795-4dcb-a600-d38dcf5af343","Type":"ContainerStarted","Data":"3e193b37e5f4ef6ae783c5ee454116ec0067611c623d18186a64df1b99eea2e6"}
Sep 19 12:47:15.734121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:15.734078    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5" podStartSLOduration=1.734063157 podStartE2EDuration="1.734063157s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:47:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:47:15.732330023 +0000 UTC m=+1319.092572961" watchObservedRunningTime="2026-09-19 12:47:15.734063157 +0000 UTC m=+1319.094306121"
Sep 19 12:47:16.064298 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:16.064215    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:16.851673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:16.851641    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:17.567353 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:17.567321    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:18.252214 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:18.252175    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:18.827809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:18.827779    2571 scope.go:117] "RemoveContainer" containerID="87b5aaaee728ba70d589a0982457cda13a50e3259681bfb895c09d0fe45f2a4e"
Sep 19 12:47:18.922411 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:18.922384    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:19.154790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:19.154704    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:19.256510 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:19.256441    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:19.535409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:19.535376    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:19.652301 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:19.652267    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:19.879717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:19.879621    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:20.565257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:20.565224    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:21.392019 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:21.391979    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:21.950585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:21.950547    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:47:22.196522 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:22.196484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:22.304175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:22.304069    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:22.443079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:22.443043    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:23.047880 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:23.047838    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:23.866777 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:23.866744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:24.562524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:24.562490    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:24.726471 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:24.726439    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:24.726664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:24.726505    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:24.741059 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:24.741030    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:24.772762 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:24.772734    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:47:25.262535 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:25.262498    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:26.059793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:26.059761    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:26.803929 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:26.803898    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:26.919013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:26.918980    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:27.099651 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.099585    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:27.292248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.292220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:27.388331 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.388249    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:27.473316 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.473285    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"]
Sep 19 12:47:27.478055 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.478038    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.480847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.480825    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-5d6124-dockercfg-qscqr\""
Sep 19 12:47:27.480969 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.480825    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-5d6124\""
Sep 19 12:47:27.488654 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.488629    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"]
Sep 19 12:47:27.569754 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.569718    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95630877-88d2-4291-a310-2c1054378869-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.569939 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.569791    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r95dn\" (UniqueName: \"kubernetes.io/projected/95630877-88d2-4291-a310-2c1054378869-kube-api-access-r95dn\") pod \"payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.670801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.670718    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95630877-88d2-4291-a310-2c1054378869-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.670956 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.670813    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r95dn\" (UniqueName: \"kubernetes.io/projected/95630877-88d2-4291-a310-2c1054378869-kube-api-access-r95dn\") pod \"payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.671602 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.671581    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95630877-88d2-4291-a310-2c1054378869-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.679862 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.679840    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r95dn\" (UniqueName: \"kubernetes.io/projected/95630877-88d2-4291-a310-2c1054378869-kube-api-access-r95dn\") pod \"payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:27.790104 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:27.790076    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:28.064298 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:28.064273    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"]
Sep 19 12:47:28.066856 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:47:28.066831    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95630877_88d2_4291_a310_2c1054378869.slice/crio-3bf31addfd6837a78ccee2598fec5e8ebb89a5230ebd9f01b5a37d8d047c91d0 WatchSource:0}: Error finding container 3bf31addfd6837a78ccee2598fec5e8ebb89a5230ebd9f01b5a37d8d047c91d0: Status 404 returned error can't find the container with id 3bf31addfd6837a78ccee2598fec5e8ebb89a5230ebd9f01b5a37d8d047c91d0
Sep 19 12:47:28.785005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:28.784966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr" event={"ID":"95630877-88d2-4291-a310-2c1054378869","Type":"ContainerStarted","Data":"ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798"}
Sep 19 12:47:28.785410 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:28.785030    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr" event={"ID":"95630877-88d2-4291-a310-2c1054378869","Type":"ContainerStarted","Data":"3bf31addfd6837a78ccee2598fec5e8ebb89a5230ebd9f01b5a37d8d047c91d0"}
Sep 19 12:47:28.785410 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:28.785124    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:28.804042 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:28.803997    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr" podStartSLOduration=1.803982899 podStartE2EDuration="1.803982899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:47:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:47:28.802986034 +0000 UTC m=+1332.163228975" watchObservedRunningTime="2026-09-19 12:47:28.803982899 +0000 UTC m=+1332.164225840"
Sep 19 12:47:39.792693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:39.792613    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 12:47:46.314410 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:46.314374    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:46.414513 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:46.414476    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:49.272423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:49.272383    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:50.058249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:50.058207    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:50.716419 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:50.716386    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:51.386778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:51.386744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:52.156936 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:52.156902    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:52.255514 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:52.255481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:52.861131 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:52.861097    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:53.666110 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:53.666071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:54.363845 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:54.363810    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:47:55.053166 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:47:55.053136    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:48:09.706854 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:09.706814    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:11.667346 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:11.667312    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:12.577153 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:12.577106    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:13.319681 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:13.319650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:13.421147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:13.421114    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:14.156483 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:14.156425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:14.288190 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:14.288161    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:14.860920 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:14.860879    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:15.664940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:15.664907    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:16.493621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:16.493591    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:17.333751 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:17.333714    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:17.547147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:17.547116    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"]
Sep 19 12:48:17.574871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:17.574841    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:18.370432 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:18.370397    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:23.301476 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:23.301426    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:26.869683 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:26.869650    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:26.959448 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:26.959420    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:27.047038 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:27.046999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:27.149846 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:27.149768    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:27.305668 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:27.305624    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:27.749518 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:27.749481    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:28.478289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:28.478252    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:28.856049 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:28.855960    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:48:29.441140 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:29.441093    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:30.277585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:30.277539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:30.487617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:30.487580    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:31.253812 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:31.253771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:32.145775 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.145739    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:32.568880 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.568839    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67" podUID="aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" containerName="main" containerID="cri-o://03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73" gracePeriod=45
Sep 19 12:48:32.815889 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.815862    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:48:32.890327 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890254    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-home\") pod \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") "
Sep 19 12:48:32.890498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890338    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-model-cache\") pod \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") "
Sep 19 12:48:32.890498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890355    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tmp-dir\") pod \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") "
Sep 19 12:48:32.890498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890378    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-dshm\") pod \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") "
Sep 19 12:48:32.890498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890408    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4hd9\" (UniqueName: \"kubernetes.io/projected/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-kube-api-access-h4hd9\") pod \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") "
Sep 19 12:48:32.890498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890427    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tls-certs\") pod \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\" (UID: \"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa\") "
Sep 19 12:48:32.890774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890547    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-home" (OuterVolumeSpecName: "home") pod "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" (UID: "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:32.890774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890630    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-model-cache" (OuterVolumeSpecName: "model-cache") pod "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" (UID: "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:32.890774 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890721    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" (UID: "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:32.890913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890836    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:32.890913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890855    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:32.890913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.890869    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:32.892675 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.892647    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-dshm" (OuterVolumeSpecName: "dshm") pod "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" (UID: "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:32.892675 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.892661    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" (UID: "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:48:32.892809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.892736    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-kube-api-access-h4hd9" (OuterVolumeSpecName: "kube-api-access-h4hd9") pod "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" (UID: "aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa"). InnerVolumeSpecName "kube-api-access-h4hd9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:48:32.988394 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.988362    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:32.992325 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.992307    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:32.992415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.992326    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4hd9\" (UniqueName: \"kubernetes.io/projected/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-kube-api-access-h4hd9\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:32.992415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:32.992336    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:33.111454 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.111418    2571 generic.go:356] "Generic (PLEG): container finished" podID="aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" containerID="03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73" exitCode=0
Sep 19 12:48:33.111454 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.111477    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67" event={"ID":"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa","Type":"ContainerDied","Data":"03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73"}
Sep 19 12:48:33.111704 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.111490    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"
Sep 19 12:48:33.111704 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.111510    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67" event={"ID":"aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa","Type":"ContainerDied","Data":"9ddc250bf8b6757c739ba5d8f182e19a61bafe615cc09e20ae85dd666d35e56b"}
Sep 19 12:48:33.111704 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.111528    2571 scope.go:117] "RemoveContainer" containerID="03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73"
Sep 19 12:48:33.120839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.120820    2571 scope.go:117] "RemoveContainer" containerID="03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73"
Sep 19 12:48:33.121128 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:48:33.121104    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73\": container with ID starting with 03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73 not found: ID does not exist" containerID="03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73"
Sep 19 12:48:33.121176 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.121139    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73"} err="failed to get container status \"03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73\": rpc error: code = NotFound desc = could not find container \"03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73\": container with ID starting with 03d583656ea9be54b9efc56d436f740d8c144245f91544fa9ea5a9a495ef3f73 not found: ID does not exist"
Sep 19 12:48:33.134215 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.134185    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"]
Sep 19 12:48:33.138834 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:33.138815    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/e2e-conflict-b-3ed226d2-kserve-579d9c8bc-xqj67"]
Sep 19 12:48:34.740915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:34.740883    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" path="/var/lib/kubelet/pods/aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa/volumes"
Sep 19 12:48:35.257844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:35.257802    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:35.258999 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:35.258978    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"]
Sep 19 12:48:35.354474 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:35.354437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:36.096539 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:36.096499    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:50.278192 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.278152    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h" podUID="de678c4e-4602-4079-b49e-05f9b26d8f7f" containerName="main" containerID="cri-o://79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768" gracePeriod=45
Sep 19 12:48:50.522392 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.522368    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:48:50.653827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.653744    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjzpl\" (UniqueName: \"kubernetes.io/projected/de678c4e-4602-4079-b49e-05f9b26d8f7f-kube-api-access-zjzpl\") pod \"de678c4e-4602-4079-b49e-05f9b26d8f7f\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") "
Sep 19 12:48:50.653827 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.653783    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de678c4e-4602-4079-b49e-05f9b26d8f7f-tls-certs\") pod \"de678c4e-4602-4079-b49e-05f9b26d8f7f\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") "
Sep 19 12:48:50.654078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.653845    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-dshm\") pod \"de678c4e-4602-4079-b49e-05f9b26d8f7f\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") "
Sep 19 12:48:50.654078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.653866    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-tmp-dir\") pod \"de678c4e-4602-4079-b49e-05f9b26d8f7f\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") "
Sep 19 12:48:50.654078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.653894    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-home\") pod \"de678c4e-4602-4079-b49e-05f9b26d8f7f\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") "
Sep 19 12:48:50.654078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.653909    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-model-cache\") pod \"de678c4e-4602-4079-b49e-05f9b26d8f7f\" (UID: \"de678c4e-4602-4079-b49e-05f9b26d8f7f\") "
Sep 19 12:48:50.654307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.654120    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de678c4e-4602-4079-b49e-05f9b26d8f7f" (UID: "de678c4e-4602-4079-b49e-05f9b26d8f7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:50.654307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.654134    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-home" (OuterVolumeSpecName: "home") pod "de678c4e-4602-4079-b49e-05f9b26d8f7f" (UID: "de678c4e-4602-4079-b49e-05f9b26d8f7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:50.654307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.654227    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:50.654307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.654244    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:50.654307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.654212    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "de678c4e-4602-4079-b49e-05f9b26d8f7f" (UID: "de678c4e-4602-4079-b49e-05f9b26d8f7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:50.656166 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.656142    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de678c4e-4602-4079-b49e-05f9b26d8f7f-kube-api-access-zjzpl" (OuterVolumeSpecName: "kube-api-access-zjzpl") pod "de678c4e-4602-4079-b49e-05f9b26d8f7f" (UID: "de678c4e-4602-4079-b49e-05f9b26d8f7f"). InnerVolumeSpecName "kube-api-access-zjzpl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:48:50.656270 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.656165    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-dshm" (OuterVolumeSpecName: "dshm") pod "de678c4e-4602-4079-b49e-05f9b26d8f7f" (UID: "de678c4e-4602-4079-b49e-05f9b26d8f7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:48:50.656270 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.656171    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de678c4e-4602-4079-b49e-05f9b26d8f7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de678c4e-4602-4079-b49e-05f9b26d8f7f" (UID: "de678c4e-4602-4079-b49e-05f9b26d8f7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:48:50.755249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.755218    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjzpl\" (UniqueName: \"kubernetes.io/projected/de678c4e-4602-4079-b49e-05f9b26d8f7f-kube-api-access-zjzpl\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:50.755249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.755243    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de678c4e-4602-4079-b49e-05f9b26d8f7f-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:50.755249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.755253    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:50.755447 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:50.755261    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de678c4e-4602-4079-b49e-05f9b26d8f7f-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:48:51.205224 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.205187    2571 generic.go:356] "Generic (PLEG): container finished" podID="de678c4e-4602-4079-b49e-05f9b26d8f7f" containerID="79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768" exitCode=0
Sep 19 12:48:51.205425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.205230    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"
Sep 19 12:48:51.205425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.205239    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h" event={"ID":"de678c4e-4602-4079-b49e-05f9b26d8f7f","Type":"ContainerDied","Data":"79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768"}
Sep 19 12:48:51.205425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.205271    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h" event={"ID":"de678c4e-4602-4079-b49e-05f9b26d8f7f","Type":"ContainerDied","Data":"d394244e794b55df7efeba696540ac4e621e6616378940d2741d3e62c5342aca"}
Sep 19 12:48:51.205425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.205286    2571 scope.go:117] "RemoveContainer" containerID="79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768"
Sep 19 12:48:51.214079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.214053    2571 scope.go:117] "RemoveContainer" containerID="79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768"
Sep 19 12:48:51.214595 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:48:51.214443    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768\": container with ID starting with 79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768 not found: ID does not exist" containerID="79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768"
Sep 19 12:48:51.214698 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.214620    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768"} err="failed to get container status \"79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768\": rpc error: code = NotFound desc = could not find container \"79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768\": container with ID starting with 79c9917f0cfa54f6100719bcbff891af37e6fc9dcbd7f94f40c5ced96c4a4768 not found: ID does not exist"
Sep 19 12:48:51.224303 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.224273    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"]
Sep 19 12:48:51.227571 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:51.227550    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e9a75910/e2e-lifecycle-model-e9a75910-kserve-5d5c7cbbf5-wdd7h"]
Sep 19 12:48:52.510734 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.510691    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"]
Sep 19 12:48:52.511213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.511055    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" containerName="main"
Sep 19 12:48:52.511213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.511067    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" containerName="main"
Sep 19 12:48:52.511213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.511084    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de678c4e-4602-4079-b49e-05f9b26d8f7f" containerName="main"
Sep 19 12:48:52.511213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.511093    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="de678c4e-4602-4079-b49e-05f9b26d8f7f" containerName="main"
Sep 19 12:48:52.511213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.511157    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="de678c4e-4602-4079-b49e-05f9b26d8f7f" containerName="main"
Sep 19 12:48:52.511213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.511172    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aadcd3b4-dd28-4dfb-a84a-fa3e94cc5aaa" containerName="main"
Sep 19 12:48:52.520278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.520246    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.523185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.523157    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"distinct-w4-04a668-kserve-self-signed-certs\""
Sep 19 12:48:52.524746 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.524721    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"]
Sep 19 12:48:52.546315 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.546290    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:48:52.670928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.670899    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-tmp-dir\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.670928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.670931    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-model-cache\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.671140 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.670956    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpjr7\" (UniqueName: \"kubernetes.io/projected/7906c6bf-e287-43d6-9463-993052a58779-kube-api-access-qpjr7\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.671140 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.670976    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-home\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.671140 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.671047    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7906c6bf-e287-43d6-9463-993052a58779-tls-certs\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.671255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.671138    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-dshm\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.739279 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.739245    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de678c4e-4602-4079-b49e-05f9b26d8f7f" path="/var/lib/kubelet/pods/de678c4e-4602-4079-b49e-05f9b26d8f7f/volumes"
Sep 19 12:48:52.772571 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772519    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-tmp-dir\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.772571 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772546    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-model-cache\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.772733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772571    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpjr7\" (UniqueName: \"kubernetes.io/projected/7906c6bf-e287-43d6-9463-993052a58779-kube-api-access-qpjr7\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.772733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772598    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-home\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.772733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772626    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7906c6bf-e287-43d6-9463-993052a58779-tls-certs\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.772733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772688    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-dshm\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.772943 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772928    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-model-cache\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.773002 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.772947    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-tmp-dir\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.773057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.773014    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-home\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.774903 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.774884    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-dshm\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.775307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.775286    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7906c6bf-e287-43d6-9463-993052a58779-tls-certs\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.781428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.781406    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpjr7\" (UniqueName: \"kubernetes.io/projected/7906c6bf-e287-43d6-9463-993052a58779-kube-api-access-qpjr7\") pod \"distinct-w4-04a668-kserve-5f495dd454-t25xh\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:52.830374 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:52.830355    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:48:53.102261 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:53.102235    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"]
Sep 19 12:48:53.103506 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:48:53.103482    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7906c6bf_e287_43d6_9463_993052a58779.slice/crio-8d5e390308e2e35341b5ce46372ecf0f95303c97da3974107e6777b75b7b8992 WatchSource:0}: Error finding container 8d5e390308e2e35341b5ce46372ecf0f95303c97da3974107e6777b75b7b8992: Status 404 returned error can't find the container with id 8d5e390308e2e35341b5ce46372ecf0f95303c97da3974107e6777b75b7b8992
Sep 19 12:48:53.218664 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:53.218629    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh" event={"ID":"7906c6bf-e287-43d6-9463-993052a58779","Type":"ContainerStarted","Data":"fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14"}
Sep 19 12:48:53.218810 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:53.218680    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh" event={"ID":"7906c6bf-e287-43d6-9463-993052a58779","Type":"ContainerStarted","Data":"8d5e390308e2e35341b5ce46372ecf0f95303c97da3974107e6777b75b7b8992"}
Sep 19 12:48:53.242152 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:53.242107    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh" podStartSLOduration=1.242092915 podStartE2EDuration="1.242092915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:48:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:48:53.240903853 +0000 UTC m=+1416.601146850" watchObservedRunningTime="2026-09-19 12:48:53.242092915 +0000 UTC m=+1416.602335855"
Sep 19 12:48:57.744738 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:48:57.744702    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:49:02.662351 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:02.662260    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:02.830915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:02.830884    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:49:02.831140 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:02.831122    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:49:02.843362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:02.843337    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:49:03.280415 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:03.280382    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:49:11.789347 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:11.789310    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:12.576358 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:12.576315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:14.056943 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:14.056909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:14.157652 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:14.157610    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:18.303751 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:18.303718    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:24.155501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:24.155439    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:32.529180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:32.529148    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:49:46.328381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.328335    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"]
Sep 19 12:49:46.331896 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.331874    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.334836 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.334818    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-04a668\"/\"distinct-2-w4-04a668-kserve-self-signed-certs\""
Sep 19 12:49:46.341133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.341112    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"]
Sep 19 12:49:46.347114 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.347091    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-model-cache\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.347203 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.347155    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-dshm\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.347249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.347204    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl8m4\" (UniqueName: \"kubernetes.io/projected/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-kube-api-access-bl8m4\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.347297 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.347276    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tmp-dir\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.347334 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.347312    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-home\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.347368 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.347333    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tls-certs\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.448620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.448590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-dshm\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.448790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.448646    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl8m4\" (UniqueName: \"kubernetes.io/projected/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-kube-api-access-bl8m4\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.448790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.448700    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tmp-dir\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.448790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.448724    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-home\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.448790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.448752    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tls-certs\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.449003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.448813    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-model-cache\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.449120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.449096    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tmp-dir\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.449209 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.449132    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-home\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.449209 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.449177    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-model-cache\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.450570 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.450531    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:46.450958 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.450942    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-dshm\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.451381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.451363    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tls-certs\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.459186 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.459165    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl8m4\" (UniqueName: \"kubernetes.io/projected/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-kube-api-access-bl8m4\") pod \"distinct-2-w4-04a668-kserve-8c675db8f-zsbt9\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") " pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.640512 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.640411    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:46.911733 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:46.911710    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"]
Sep 19 12:49:46.913327 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:49:46.913303    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf74ed2f_6bcb_4c47_aea9_c81ab19b05a8.slice/crio-474a7d5e650af95a3b9e7fc3605a2b13c5fd1434ffce25dc2ed59707c80bb7d8 WatchSource:0}: Error finding container 474a7d5e650af95a3b9e7fc3605a2b13c5fd1434ffce25dc2ed59707c80bb7d8: Status 404 returned error can't find the container with id 474a7d5e650af95a3b9e7fc3605a2b13c5fd1434ffce25dc2ed59707c80bb7d8
Sep 19 12:49:47.492840 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:47.492809    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9" event={"ID":"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8","Type":"ContainerStarted","Data":"c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9"}
Sep 19 12:49:47.493197 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:47.492851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9" event={"ID":"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8","Type":"ContainerStarted","Data":"474a7d5e650af95a3b9e7fc3605a2b13c5fd1434ffce25dc2ed59707c80bb7d8"}
Sep 19 12:49:47.512912 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:47.512862    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9" podStartSLOduration=1.512848001 podStartE2EDuration="1.512848001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:49:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:49:47.510387758 +0000 UTC m=+1470.870630701" watchObservedRunningTime="2026-09-19 12:49:47.512848001 +0000 UTC m=+1470.873090989"
Sep 19 12:49:49.982036 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:49.981997    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:50.088073 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:50.088034    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:50.250955 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:50.250863    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:51.431590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:51.431558    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:51.534705 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:51.534678    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:51.631741 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:51.631712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:52.261233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:52.261197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:52.358519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:52.358488    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:52.463134 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:52.463106    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:53.657357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:53.657322    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:49:56.641108 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:56.641075    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:56.641590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:56.641123    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:56.653550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:56.653529    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:57.555639 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:57.555613    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:49:58.343164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:49:58.343133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:00.263492 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:00.263439    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:00.333229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:00.333197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:03.731941 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:03.731907    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:50:05.923537 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:05.923500    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:06.030472 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:06.030437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:06.129005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:06.128975    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:16.611887 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:16.611853    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:16.892589 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:16.892509    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:32.388227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:32.388143    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:37.740843 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:37.740808    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:41.232326 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:41.232291    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:50:43.482132 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:43.482092    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:43.548450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:43.548418    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:43.804779 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:43.804693    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:44.104887 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:44.104799    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:44.794004 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:44.793970    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:47.156403 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:47.156370    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:47.454390 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:47.454347    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:47.546488 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:47.546443    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:48.320694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:48.320662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:49.129933 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:49.129893    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:50.057258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:50.057212    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:50.765401 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:50.765357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:51.433431 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:51.433401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:52.291710 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.291674    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:52.391637 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.391603    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:52.492127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.492088    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:52.794274 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.794232    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:52.870232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.870197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:50:52.929622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.929592    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"]
Sep 19 12:50:52.932916 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.932900    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:52.935588 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.935571    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-49e81a-dockercfg-bxtxw\""
Sep 19 12:50:52.935642 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.935591    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-49e81a\""
Sep 19 12:50:52.943131 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:52.943105    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"]
Sep 19 12:50:53.019948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.019911    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e5fd8e0-4321-4508-98ca-5bcd2769b322-plugins-config-volume\") pod \"payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") " pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.020146 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.019979    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwpgg\" (UniqueName: \"kubernetes.io/projected/3e5fd8e0-4321-4508-98ca-5bcd2769b322-kube-api-access-rwpgg\") pod \"payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") " pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.121277 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.121190    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e5fd8e0-4321-4508-98ca-5bcd2769b322-plugins-config-volume\") pod \"payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") " pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.121277 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.121251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwpgg\" (UniqueName: \"kubernetes.io/projected/3e5fd8e0-4321-4508-98ca-5bcd2769b322-kube-api-access-rwpgg\") pod \"payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") " pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.121913 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.121894    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e5fd8e0-4321-4508-98ca-5bcd2769b322-plugins-config-volume\") pod \"payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") " pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.129790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.129768    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwpgg\" (UniqueName: \"kubernetes.io/projected/3e5fd8e0-4321-4508-98ca-5bcd2769b322-kube-api-access-rwpgg\") pod \"payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") " pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.243876 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.243827    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.518424 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.518397    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"]
Sep 19 12:50:53.520291 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:50:53.520257    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e5fd8e0_4321_4508_98ca_5bcd2769b322.slice/crio-ccca4129c4ddcd71770a0454c60a9a380c4c62b2ca1c339a6a93571da13c2a33 WatchSource:0}: Error finding container ccca4129c4ddcd71770a0454c60a9a380c4c62b2ca1c339a6a93571da13c2a33: Status 404 returned error can't find the container with id ccca4129c4ddcd71770a0454c60a9a380c4c62b2ca1c339a6a93571da13c2a33
Sep 19 12:50:53.825367 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.825277    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j" event={"ID":"3e5fd8e0-4321-4508-98ca-5bcd2769b322","Type":"ContainerStarted","Data":"c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372"}
Sep 19 12:50:53.825367 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.825359    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j" event={"ID":"3e5fd8e0-4321-4508-98ca-5bcd2769b322","Type":"ContainerStarted","Data":"ccca4129c4ddcd71770a0454c60a9a380c4c62b2ca1c339a6a93571da13c2a33"}
Sep 19 12:50:53.825586 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.825420    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:50:53.842735 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:50:53.842684    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j" podStartSLOduration=1.842669367 podStartE2EDuration="1.842669367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:50:53.841169993 +0000 UTC m=+1537.201412934" watchObservedRunningTime="2026-09-19 12:50:53.842669367 +0000 UTC m=+1537.202912307"
Sep 19 12:51:01.523849 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:01.523815    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:01.624030 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:01.623999    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:04.832584 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:04.832556    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:51:11.760976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:11.760929    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:12.221929 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:12.221894    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:13.167976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:13.167933    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:13.997950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:13.997911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:14.677656 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:14.677626    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:15.642363 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:15.642324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:16.499855 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:16.499820    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:16.745719 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:16.745687    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"]
Sep 19 12:51:16.753716 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:16.753648    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"]
Sep 19 12:51:16.773564 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:16.773538    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"]
Sep 19 12:51:16.778473 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:16.778440    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"]
Sep 19 12:51:16.837658 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:16.837623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:17.296126 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:17.296083    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:17.394846 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:17.394807    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:17.786097 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:17.786059    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:18.054007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.053919    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"]
Sep 19 12:51:18.057722 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.057703    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.060501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.060477    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 12:51:18.060817 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.060790    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-513a51-openshift-default-dockercfg-cg4wr\""
Sep 19 12:51:18.068422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.068394    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"]
Sep 19 12:51:18.076688 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.076651    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:18.150476 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.150423    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:18.160628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160585    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-credential-socket\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160647    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-envoy\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160691    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-podinfo\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160717    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjjvm\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-kube-api-access-wjjvm\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160766 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160741    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-token\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160765    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-certs\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160812    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-socket\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160845    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-data\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.160914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.160874    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52bb0c84-0066-4c1d-be3b-58366c6c286b-istiod-ca-cert\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262217 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-credential-socket\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262217 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262222    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-envoy\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262262    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-podinfo\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262281    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjjvm\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-kube-api-access-wjjvm\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262298    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-token\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262317    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-certs\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262333    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-socket\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-data\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262505 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262390    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52bb0c84-0066-4c1d-be3b-58366c6c286b-istiod-ca-cert\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262854 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262664    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-credential-socket\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262854 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262780    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-certs\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.262984 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262961    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-data\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.263027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.262988    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-socket\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.263092 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.263076    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52bb0c84-0066-4c1d-be3b-58366c6c286b-istiod-ca-cert\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.264683 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.264660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-envoy\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.264885 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.264868    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-podinfo\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.270283 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.270265    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-token\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.270494 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.270474    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjjvm\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-kube-api-access-wjjvm\") pod \"e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") " pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.288138 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.288114    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:18.369216 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.369129    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:18.648573 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.648545    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"]
Sep 19 12:51:18.650186 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:51:18.650159    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52bb0c84_0066_4c1d_be3b_58366c6c286b.slice/crio-65609d2e3b57ffc699229bedc524bcb48ef0f612b3b81370a7bbc3c57db7acf0 WatchSource:0}: Error finding container 65609d2e3b57ffc699229bedc524bcb48ef0f612b3b81370a7bbc3c57db7acf0: Status 404 returned error can't find the container with id 65609d2e3b57ffc699229bedc524bcb48ef0f612b3b81370a7bbc3c57db7acf0
Sep 19 12:51:18.652667 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.652634    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:51:18.652742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.652698    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:51:18.652742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.652726    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:51:18.954778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.954746    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" event={"ID":"52bb0c84-0066-4c1d-be3b-58366c6c286b","Type":"ContainerStarted","Data":"b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"}
Sep 19 12:51:18.955151 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.954799    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" event={"ID":"52bb0c84-0066-4c1d-be3b-58366c6c286b","Type":"ContainerStarted","Data":"65609d2e3b57ffc699229bedc524bcb48ef0f612b3b81370a7bbc3c57db7acf0"}
Sep 19 12:51:18.975915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:18.975869    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podStartSLOduration=0.975854474 podStartE2EDuration="975.854474ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:51:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:51:18.975165686 +0000 UTC m=+1562.335408628" watchObservedRunningTime="2026-09-19 12:51:18.975854474 +0000 UTC m=+1562.336097475"
Sep 19 12:51:19.055491 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:19.055437    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:19.370170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:19.370076    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:19.375626 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:19.375600    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:19.758593 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:19.758556    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:19.960278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:19.960253    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:19.961380 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:19.961366    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:51:20.461230 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:20.461195    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:21.035929 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:21.035884    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:21.131235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:21.131194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:21.546718 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:21.546675    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:22.454206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:22.454161    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:23.184548 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:23.184512    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:23.275874 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:23.275837    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:23.509870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:23.509831    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:23.757161 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:23.757129    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:23.864779 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:23.864696    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:24.347824 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:24.347783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:25.268793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:25.268752    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:25.987214 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:25.987181    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:26.672440 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:26.672407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:27.476302 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:27.476259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:29.183350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.183318    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"]
Sep 19 12:51:29.186826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.186781    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:29.187937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.187917    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-a2e924-29830365-5p524"]
Sep 19 12:51:29.291049 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.291019    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:29.394205 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.394168    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:29.491599 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.491569    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:29.809998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.809916    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:29.927162 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:29.927131    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:30.740133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:30.740092    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c8f82ea-de49-46c0-b200-6334ef6fb188" path="/var/lib/kubelet/pods/3c8f82ea-de49-46c0-b200-6334ef6fb188/volumes"
Sep 19 12:51:30.901130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:30.901096    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:31.672100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:31.672067    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:31.765501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:31.765439    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5" podUID="c7c386f9-b795-4dcb-a600-d38dcf5af343" containerName="main" containerID="cri-o://50e52d0a2f2642bdf9e3a642c482a838d474feeea2a0cc7b72456e1f75a9044d" gracePeriod=45
Sep 19 12:51:31.772581 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:31.772548    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92" podUID="7533bbc7-efa7-438f-843f-9ea8142e8701" containerName="main" containerID="cri-o://9ee46549352c15d10503132f71405b0e54436d361d0bea97b4ed0f7cafe3bd89" gracePeriod=45
Sep 19 12:51:31.791934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:31.791903    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw" podUID="a47b9066-4012-4ba0-84a4-e37832db3adf" containerName="main" containerID="cri-o://743e14ec9d5f445269515ac106d167d6db950cea4fedd072dfcb191b1d058262" gracePeriod=45
Sep 19 12:51:31.796532 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:31.796507    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6" podUID="51fe2b4a-6611-42dd-95a5-f4834e5a7caa" containerName="main" containerID="cri-o://ff7a1a3d235b0226c31305e2755ae8bf676b7ee7bcecdaafa9f68ab1f0db7301" gracePeriod=45
Sep 19 12:51:32.025159 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.025124    2571 generic.go:356] "Generic (PLEG): container finished" podID="a47b9066-4012-4ba0-84a4-e37832db3adf" containerID="743e14ec9d5f445269515ac106d167d6db950cea4fedd072dfcb191b1d058262" exitCode=0
Sep 19 12:51:32.025337 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.025199    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw" event={"ID":"a47b9066-4012-4ba0-84a4-e37832db3adf","Type":"ContainerDied","Data":"743e14ec9d5f445269515ac106d167d6db950cea4fedd072dfcb191b1d058262"}
Sep 19 12:51:32.027224 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.027069    2571 generic.go:356] "Generic (PLEG): container finished" podID="51fe2b4a-6611-42dd-95a5-f4834e5a7caa" containerID="ff7a1a3d235b0226c31305e2755ae8bf676b7ee7bcecdaafa9f68ab1f0db7301" exitCode=0
Sep 19 12:51:32.027224 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.027143    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6" event={"ID":"51fe2b4a-6611-42dd-95a5-f4834e5a7caa","Type":"ContainerDied","Data":"ff7a1a3d235b0226c31305e2755ae8bf676b7ee7bcecdaafa9f68ab1f0db7301"}
Sep 19 12:51:32.029027 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.029003    2571 generic.go:356] "Generic (PLEG): container finished" podID="7533bbc7-efa7-438f-843f-9ea8142e8701" containerID="9ee46549352c15d10503132f71405b0e54436d361d0bea97b4ed0f7cafe3bd89" exitCode=0
Sep 19 12:51:32.029145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.029073    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92" event={"ID":"7533bbc7-efa7-438f-843f-9ea8142e8701","Type":"ContainerDied","Data":"9ee46549352c15d10503132f71405b0e54436d361d0bea97b4ed0f7cafe3bd89"}
Sep 19 12:51:32.030924 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.030902    2571 generic.go:356] "Generic (PLEG): container finished" podID="c7c386f9-b795-4dcb-a600-d38dcf5af343" containerID="50e52d0a2f2642bdf9e3a642c482a838d474feeea2a0cc7b72456e1f75a9044d" exitCode=0
Sep 19 12:51:32.031015 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.030951    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5" event={"ID":"c7c386f9-b795-4dcb-a600-d38dcf5af343","Type":"ContainerDied","Data":"50e52d0a2f2642bdf9e3a642c482a838d474feeea2a0cc7b72456e1f75a9044d"}
Sep 19 12:51:32.079196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.079175    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:51:32.083168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.083148    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:51:32.115126 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.115100    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:51:32.118551 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.118530    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:51:32.192103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192074    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7533bbc7-efa7-438f-843f-9ea8142e8701-tls-certs\") pod \"7533bbc7-efa7-438f-843f-9ea8142e8701\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192111    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7v2rk\" (UniqueName: \"kubernetes.io/projected/c7c386f9-b795-4dcb-a600-d38dcf5af343-kube-api-access-7v2rk\") pod \"c7c386f9-b795-4dcb-a600-d38dcf5af343\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192134    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npzt6\" (UniqueName: \"kubernetes.io/projected/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-kube-api-access-npzt6\") pod \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192155    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-tmp-dir\") pod \"c7c386f9-b795-4dcb-a600-d38dcf5af343\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192178    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-dshm\") pod \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192199    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-model-cache\") pod \"c7c386f9-b795-4dcb-a600-d38dcf5af343\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192230    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-dshm\") pod \"a47b9066-4012-4ba0-84a4-e37832db3adf\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") "
Sep 19 12:51:32.192257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192257    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-tmp-dir\") pod \"a47b9066-4012-4ba0-84a4-e37832db3adf\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") "
Sep 19 12:51:32.192828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192797    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a47b9066-4012-4ba0-84a4-e37832db3adf" (UID: "a47b9066-4012-4ba0-84a4-e37832db3adf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.192944 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192287    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-model-cache\") pod \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") "
Sep 19 12:51:32.192944 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192928    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7c386f9-b795-4dcb-a600-d38dcf5af343" (UID: "c7c386f9-b795-4dcb-a600-d38dcf5af343"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.193067 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.192984    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-dshm\") pod \"c7c386f9-b795-4dcb-a600-d38dcf5af343\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") "
Sep 19 12:51:32.193067 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193044    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tmp-dir\") pod \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") "
Sep 19 12:51:32.193168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193093    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-tmp-dir\") pod \"7533bbc7-efa7-438f-843f-9ea8142e8701\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") "
Sep 19 12:51:32.193168 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193133    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b9066-4012-4ba0-84a4-e37832db3adf-tls-certs\") pod \"a47b9066-4012-4ba0-84a4-e37832db3adf\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") "
Sep 19 12:51:32.193261 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193166    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-home\") pod \"c7c386f9-b795-4dcb-a600-d38dcf5af343\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") "
Sep 19 12:51:32.193261 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193181    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-model-cache" (OuterVolumeSpecName: "model-cache") pod "51fe2b4a-6611-42dd-95a5-f4834e5a7caa" (UID: "51fe2b4a-6611-42dd-95a5-f4834e5a7caa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.193362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193271    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7c386f9-b795-4dcb-a600-d38dcf5af343" (UID: "c7c386f9-b795-4dcb-a600-d38dcf5af343"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.193362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193347    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-home" (OuterVolumeSpecName: "home") pod "c7c386f9-b795-4dcb-a600-d38dcf5af343" (UID: "c7c386f9-b795-4dcb-a600-d38dcf5af343"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.193948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.193852    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "51fe2b4a-6611-42dd-95a5-f4834e5a7caa" (UID: "51fe2b4a-6611-42dd-95a5-f4834e5a7caa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.194799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.194774    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-home\") pod \"a47b9066-4012-4ba0-84a4-e37832db3adf\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") "
Sep 19 12:51:32.202812 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.202790    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mwht\" (UniqueName: \"kubernetes.io/projected/7533bbc7-efa7-438f-843f-9ea8142e8701-kube-api-access-8mwht\") pod \"7533bbc7-efa7-438f-843f-9ea8142e8701\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") "
Sep 19 12:51:32.202933 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.195265    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-home" (OuterVolumeSpecName: "home") pod "a47b9066-4012-4ba0-84a4-e37832db3adf" (UID: "a47b9066-4012-4ba0-84a4-e37832db3adf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.195558    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-kube-api-access-npzt6" (OuterVolumeSpecName: "kube-api-access-npzt6") pod "51fe2b4a-6611-42dd-95a5-f4834e5a7caa" (UID: "51fe2b4a-6611-42dd-95a5-f4834e5a7caa"). InnerVolumeSpecName "kube-api-access-npzt6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.195673    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-dshm" (OuterVolumeSpecName: "dshm") pod "51fe2b4a-6611-42dd-95a5-f4834e5a7caa" (UID: "51fe2b4a-6611-42dd-95a5-f4834e5a7caa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.195700    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-dshm" (OuterVolumeSpecName: "dshm") pod "a47b9066-4012-4ba0-84a4-e37832db3adf" (UID: "a47b9066-4012-4ba0-84a4-e37832db3adf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.197476    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7c386f9-b795-4dcb-a600-d38dcf5af343-kube-api-access-7v2rk" (OuterVolumeSpecName: "kube-api-access-7v2rk") pod "c7c386f9-b795-4dcb-a600-d38dcf5af343" (UID: "c7c386f9-b795-4dcb-a600-d38dcf5af343"). InnerVolumeSpecName "kube-api-access-7v2rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.198319    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7533bbc7-efa7-438f-843f-9ea8142e8701-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7533bbc7-efa7-438f-843f-9ea8142e8701" (UID: "7533bbc7-efa7-438f-843f-9ea8142e8701"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.198336    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-dshm" (OuterVolumeSpecName: "dshm") pod "c7c386f9-b795-4dcb-a600-d38dcf5af343" (UID: "c7c386f9-b795-4dcb-a600-d38dcf5af343"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.199864    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a47b9066-4012-4ba0-84a4-e37832db3adf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a47b9066-4012-4ba0-84a4-e37832db3adf" (UID: "a47b9066-4012-4ba0-84a4-e37832db3adf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:51:32.202998 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.202864    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7533bbc7-efa7-438f-843f-9ea8142e8701" (UID: "7533bbc7-efa7-438f-843f-9ea8142e8701"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.203338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.202998    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-home\") pod \"7533bbc7-efa7-438f-843f-9ea8142e8701\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") "
Sep 19 12:51:32.203338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203060    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c386f9-b795-4dcb-a600-d38dcf5af343-tls-certs\") pod \"c7c386f9-b795-4dcb-a600-d38dcf5af343\" (UID: \"c7c386f9-b795-4dcb-a600-d38dcf5af343\") "
Sep 19 12:51:32.203338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203148    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-home" (OuterVolumeSpecName: "home") pod "7533bbc7-efa7-438f-843f-9ea8142e8701" (UID: "7533bbc7-efa7-438f-843f-9ea8142e8701"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.203338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203168    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-dshm\") pod \"7533bbc7-efa7-438f-843f-9ea8142e8701\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") "
Sep 19 12:51:32.203616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203499    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-home\") pod \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") "
Sep 19 12:51:32.203616 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203536    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-model-cache\") pod \"7533bbc7-efa7-438f-843f-9ea8142e8701\" (UID: \"7533bbc7-efa7-438f-843f-9ea8142e8701\") "
Sep 19 12:51:32.203760 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203613    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-home" (OuterVolumeSpecName: "home") pod "51fe2b4a-6611-42dd-95a5-f4834e5a7caa" (UID: "51fe2b4a-6611-42dd-95a5-f4834e5a7caa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.203849 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203763    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-model-cache\") pod \"a47b9066-4012-4ba0-84a4-e37832db3adf\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") "
Sep 19 12:51:32.203849 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203821    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tls-certs\") pod \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\" (UID: \"51fe2b4a-6611-42dd-95a5-f4834e5a7caa\") "
Sep 19 12:51:32.204013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203856    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxl4p\" (UniqueName: \"kubernetes.io/projected/a47b9066-4012-4ba0-84a4-e37832db3adf-kube-api-access-zxl4p\") pod \"a47b9066-4012-4ba0-84a4-e37832db3adf\" (UID: \"a47b9066-4012-4ba0-84a4-e37832db3adf\") "
Sep 19 12:51:32.204013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203914    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-model-cache" (OuterVolumeSpecName: "model-cache") pod "a47b9066-4012-4ba0-84a4-e37832db3adf" (UID: "a47b9066-4012-4ba0-84a4-e37832db3adf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.204013 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.203985    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-model-cache" (OuterVolumeSpecName: "model-cache") pod "7533bbc7-efa7-438f-843f-9ea8142e8701" (UID: "7533bbc7-efa7-438f-843f-9ea8142e8701"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.204322 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204288    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204322 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204309    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204322 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204322    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204334    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204348    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204360    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7533bbc7-efa7-438f-843f-9ea8142e8701-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204404    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7v2rk\" (UniqueName: \"kubernetes.io/projected/c7c386f9-b795-4dcb-a600-d38dcf5af343-kube-api-access-7v2rk\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204418    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npzt6\" (UniqueName: \"kubernetes.io/projected/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-kube-api-access-npzt6\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204431    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204443    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204477    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204489    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204501    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b9066-4012-4ba0-84a4-e37832db3adf-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204513    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204524    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204535    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204546    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204567    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b9066-4012-4ba0-84a4-e37832db3adf-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.204591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204578    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7c386f9-b795-4dcb-a600-d38dcf5af343-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.205105 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.204881    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7533bbc7-efa7-438f-843f-9ea8142e8701-kube-api-access-8mwht" (OuterVolumeSpecName: "kube-api-access-8mwht") pod "7533bbc7-efa7-438f-843f-9ea8142e8701" (UID: "7533bbc7-efa7-438f-843f-9ea8142e8701"). InnerVolumeSpecName "kube-api-access-8mwht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:32.205500 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.205452    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7c386f9-b795-4dcb-a600-d38dcf5af343-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7c386f9-b795-4dcb-a600-d38dcf5af343" (UID: "c7c386f9-b795-4dcb-a600-d38dcf5af343"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:51:32.206057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.206036    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a47b9066-4012-4ba0-84a4-e37832db3adf-kube-api-access-zxl4p" (OuterVolumeSpecName: "kube-api-access-zxl4p") pod "a47b9066-4012-4ba0-84a4-e37832db3adf" (UID: "a47b9066-4012-4ba0-84a4-e37832db3adf"). InnerVolumeSpecName "kube-api-access-zxl4p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:51:32.206381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.206359    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "51fe2b4a-6611-42dd-95a5-f4834e5a7caa" (UID: "51fe2b4a-6611-42dd-95a5-f4834e5a7caa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:51:32.206743 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.206725    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-dshm" (OuterVolumeSpecName: "dshm") pod "7533bbc7-efa7-438f-843f-9ea8142e8701" (UID: "7533bbc7-efa7-438f-843f-9ea8142e8701"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:51:32.305994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.305938    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mwht\" (UniqueName: \"kubernetes.io/projected/7533bbc7-efa7-438f-843f-9ea8142e8701-kube-api-access-8mwht\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.305994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.305959    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7c386f9-b795-4dcb-a600-d38dcf5af343-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.305994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.305969    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7533bbc7-efa7-438f-843f-9ea8142e8701-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.305994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.305976    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/51fe2b4a-6611-42dd-95a5-f4834e5a7caa-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.305994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.305985    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxl4p\" (UniqueName: \"kubernetes.io/projected/a47b9066-4012-4ba0-84a4-e37832db3adf-kube-api-access-zxl4p\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:51:32.518090 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:32.518060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:33.037915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.037837    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"
Sep 19 12:51:33.037915 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.037832    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92" event={"ID":"7533bbc7-efa7-438f-843f-9ea8142e8701","Type":"ContainerDied","Data":"3f7273215cca58a274f26f489b5e514b4650441a350487d3e5368c82f2fb0876"}
Sep 19 12:51:33.038420 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.037952    2571 scope.go:117] "RemoveContainer" containerID="9ee46549352c15d10503132f71405b0e54436d361d0bea97b4ed0f7cafe3bd89"
Sep 19 12:51:33.039334 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.039321    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"
Sep 19 12:51:33.039406 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.039341    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5" event={"ID":"c7c386f9-b795-4dcb-a600-d38dcf5af343","Type":"ContainerDied","Data":"3e193b37e5f4ef6ae783c5ee454116ec0067611c623d18186a64df1b99eea2e6"}
Sep 19 12:51:33.040847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.040829    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"
Sep 19 12:51:33.040847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.040839    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw" event={"ID":"a47b9066-4012-4ba0-84a4-e37832db3adf","Type":"ContainerDied","Data":"080c6853a65e598d74f02b692abc5d4aaa620e0ac65166f2253957b2f54eaf82"}
Sep 19 12:51:33.042598 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.042578    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6" event={"ID":"51fe2b4a-6611-42dd-95a5-f4834e5a7caa","Type":"ContainerDied","Data":"e14bb234ec4f0432a6caa9bd7e7816003dfff283a0c56d4b3d66a6b99f8dfc08"}
Sep 19 12:51:33.042690 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.042595    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"
Sep 19 12:51:33.048088 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.048072    2571 scope.go:117] "RemoveContainer" containerID="50e52d0a2f2642bdf9e3a642c482a838d474feeea2a0cc7b72456e1f75a9044d"
Sep 19 12:51:33.055449 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.055433    2571 scope.go:117] "RemoveContainer" containerID="743e14ec9d5f445269515ac106d167d6db950cea4fedd072dfcb191b1d058262"
Sep 19 12:51:33.063041 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.063025    2571 scope.go:117] "RemoveContainer" containerID="ff7a1a3d235b0226c31305e2755ae8bf676b7ee7bcecdaafa9f68ab1f0db7301"
Sep 19 12:51:33.064587 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.064571    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"]
Sep 19 12:51:33.068560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.068541    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-2-w0-a2e924-kserve-99d78cbd7-g82l5"]
Sep 19 12:51:33.078712 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.078688    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"]
Sep 19 12:51:33.083271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.083253    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/premium-w0-a2e924-kserve-5c7f655dcc-q8jhw"]
Sep 19 12:51:33.094942 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.094921    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"]
Sep 19 12:51:33.098667 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.098649    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/distinct-w0-a2e924-kserve-ddd995fbd-mkg92"]
Sep 19 12:51:33.115764 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.115744    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"]
Sep 19 12:51:33.119519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.119494    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-a2e924/simulator-w0-a2e924-kserve-7b8984fb68-fvtc6"]
Sep 19 12:51:33.381489 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:33.381379    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:34.247892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:34.247857    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:34.741504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:34.741447    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51fe2b4a-6611-42dd-95a5-f4834e5a7caa" path="/var/lib/kubelet/pods/51fe2b4a-6611-42dd-95a5-f4834e5a7caa/volumes"
Sep 19 12:51:34.741957 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:34.741936    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7533bbc7-efa7-438f-843f-9ea8142e8701" path="/var/lib/kubelet/pods/7533bbc7-efa7-438f-843f-9ea8142e8701/volumes"
Sep 19 12:51:34.742381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:34.742362    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a47b9066-4012-4ba0-84a4-e37832db3adf" path="/var/lib/kubelet/pods/a47b9066-4012-4ba0-84a4-e37832db3adf/volumes"
Sep 19 12:51:34.743073 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:34.743052    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7c386f9-b795-4dcb-a600-d38dcf5af343" path="/var/lib/kubelet/pods/c7c386f9-b795-4dcb-a600-d38dcf5af343/volumes"
Sep 19 12:51:35.048639 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:35.048559    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:35.855795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:35.855757    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:36.263703 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:36.263667    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:36.367436 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:36.367401    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:36.551414 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:36.551298    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:37.395908 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:37.395861    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:38.260252 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:38.260215    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:38.953222 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:38.953181    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:39.723426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:39.723386    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:40.574937 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:40.574905    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:40.812993 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:40.812948    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:41.474063 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:41.474026    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:42.203602 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:42.203561    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:42.973190 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:42.973154    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:44.048133 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:44.048099    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:44.872175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:44.872139    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:45.552776 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:45.552743    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:46.220911 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:46.220881    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:47.163692 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:47.163663    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:47.989879 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:47.989849    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:48.743078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:48.743047    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:49.411051 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:49.411015    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:53.651514 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:53.651484    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:53.825496 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:53.825438    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:53.891054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:53.891022    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:54.402772 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:54.402742    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:58.334834 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:58.334797    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:51:59.497570 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.497532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:59.529549 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.529520    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"]
Sep 19 12:51:59.529832 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.529803    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j" podUID="3e5fd8e0-4321-4508-98ca-5bcd2769b322" containerName="payload-processing" containerID="cri-o://c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372" gracePeriod=30
Sep 19 12:51:59.577533 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.577499    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:59.684392 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.684343    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:59.746008 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.745973    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:51:59.771759 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.771737    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:51:59.962753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.962721    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e5fd8e0-4321-4508-98ca-5bcd2769b322-plugins-config-volume\") pod \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") "
Sep 19 12:51:59.962936 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.962768    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwpgg\" (UniqueName: \"kubernetes.io/projected/3e5fd8e0-4321-4508-98ca-5bcd2769b322-kube-api-access-rwpgg\") pod \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\" (UID: \"3e5fd8e0-4321-4508-98ca-5bcd2769b322\") "
Sep 19 12:51:59.963134 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.963109    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e5fd8e0-4321-4508-98ca-5bcd2769b322-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3e5fd8e0-4321-4508-98ca-5bcd2769b322" (UID: "3e5fd8e0-4321-4508-98ca-5bcd2769b322"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:51:59.964838 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:51:59.964819    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e5fd8e0-4321-4508-98ca-5bcd2769b322-kube-api-access-rwpgg" (OuterVolumeSpecName: "kube-api-access-rwpgg") pod "3e5fd8e0-4321-4508-98ca-5bcd2769b322" (UID: "3e5fd8e0-4321-4508-98ca-5bcd2769b322"). InnerVolumeSpecName "kube-api-access-rwpgg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:00.064450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.064354    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e5fd8e0-4321-4508-98ca-5bcd2769b322-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:00.064450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.064379    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwpgg\" (UniqueName: \"kubernetes.io/projected/3e5fd8e0-4321-4508-98ca-5bcd2769b322-kube-api-access-rwpgg\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:00.065426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.065399    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:00.164830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.164794    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:00.182592 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.182555    2571 generic.go:356] "Generic (PLEG): container finished" podID="3e5fd8e0-4321-4508-98ca-5bcd2769b322" containerID="c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372" exitCode=0
Sep 19 12:52:00.182784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.182612    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"
Sep 19 12:52:00.182784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.182660    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j" event={"ID":"3e5fd8e0-4321-4508-98ca-5bcd2769b322","Type":"ContainerDied","Data":"c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372"}
Sep 19 12:52:00.182784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.182713    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j" event={"ID":"3e5fd8e0-4321-4508-98ca-5bcd2769b322","Type":"ContainerDied","Data":"ccca4129c4ddcd71770a0454c60a9a380c4c62b2ca1c339a6a93571da13c2a33"}
Sep 19 12:52:00.182784 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.182744    2571 scope.go:117] "RemoveContainer" containerID="c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372"
Sep 19 12:52:00.192579 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.192564    2571 scope.go:117] "RemoveContainer" containerID="c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372"
Sep 19 12:52:00.192853 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:52:00.192833    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372\": container with ID starting with c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372 not found: ID does not exist" containerID="c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372"
Sep 19 12:52:00.192929 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.192858    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372"} err="failed to get container status \"c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372\": rpc error: code = NotFound desc = could not find container \"c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372\": container with ID starting with c5ff98fab31c9aeb8e5007342817e3626329f1574569c4862c99dfaa99ced372 not found: ID does not exist"
Sep 19 12:52:00.207422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.207401    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"]
Sep 19 12:52:00.210944 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.210922    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-49e81a-5966f86c9f-bp54j"]
Sep 19 12:52:00.742474 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.742415    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e5fd8e0-4321-4508-98ca-5bcd2769b322" path="/var/lib/kubelet/pods/3e5fd8e0-4321-4508-98ca-5bcd2769b322/volumes"
Sep 19 12:52:00.955599 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:00.955563    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:01.733569 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:01.733532    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:02.863381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:02.863342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:03.549204 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:03.549172    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:04.378779 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:04.378739    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:05.138756 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:05.138720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:06.603381 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:06.603339    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:06.672909 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:06.672872    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:06.846795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:06.846759    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:07.464841 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:07.464803    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:08.262877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:08.262840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:08.960747 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:08.960712    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:09.232591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.232510    2571 generic.go:356] "Generic (PLEG): container finished" podID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerID="b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b" exitCode=0
Sep 19 12:52:09.232591 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.232569    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" event={"ID":"52bb0c84-0066-4c1d-be3b-58366c6c286b","Type":"ContainerDied","Data":"b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"}
Sep 19 12:52:09.232985 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.232971    2571 scope.go:117] "RemoveContainer" containerID="b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"
Sep 19 12:52:09.234171 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.234138    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:52:09.234257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.234217    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:52:09.234257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.234251    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:52:09.741991 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:09.741944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:10.245799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:10.245765    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" event={"ID":"52bb0c84-0066-4c1d-be3b-58366c6c286b","Type":"ContainerStarted","Data":"d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785"}
Sep 19 12:52:10.369330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:10.369293    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:52:10.369964 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:10.369941    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:52:10.370062 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:10.369995    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 19 12:52:10.633487 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:10.633367    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:11.369556 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:11.369523    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:52:11.369945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:11.369585    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 19 12:52:11.436762 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:11.436720    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:12.251742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:12.251700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:12.369745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:12.369707    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:52:12.370123 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:12.369776    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 19 12:52:13.061172 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:13.061134    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:13.370482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:13.370376    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:52:13.370482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:13.370432    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 19 12:52:14.370531 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:14.370491    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:52:14.371097 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:14.370555    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 19 12:52:15.370296 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:15.370259    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 19 12:52:15.370519 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:15.370318    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 19 12:52:15.551403 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:15.551360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:16.373801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:16.373773    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:52:16.374237 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:16.374223    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:52:16.374886 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:16.374865    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:52:17.356838 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:17.356798    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:18.891720 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:18.891690    2571 scope.go:117] "RemoveContainer" containerID="0bb4126db37de3fc31904e627c73b7a777aecf495a389bfd736eda12754a2319"
Sep 19 12:52:23.102977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.102943    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:23.178958 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.178923    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:23.241495 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.241447    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:23.300891 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.300854    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:23.363184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.363098    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:23.647151 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.647071    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:23.766014 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:23.765982    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:24.458450 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:24.458413    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:25.288180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:25.288149    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:26.079289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:26.079255    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:26.871074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:26.871040    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:27.570520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:27.570483    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:28.471890 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:28.471848    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:28.942196 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:28.942116    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:52:29.581473 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:29.581436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:29.681520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:29.681491    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:29.806852 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:29.806818    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"]
Sep 19 12:52:29.807095 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:29.807071    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" containerID="cri-o://d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785" gracePeriod=30
Sep 19 12:52:29.824372 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:29.824329    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:30.481443 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:30.481393    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:31.373156 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:31.373121    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:31.376524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:31.376500    2571 patch_prober.go:28] interesting pod/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 19 12:52:31.376674 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:31.376559    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 19 12:52:32.080043 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:32.080012    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:32.843293 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:32.843259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:33.596617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:33.596581    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:33.857310 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:33.857222    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:34.529408 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:34.529377    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:34.984727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:34.984692    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:35.048247 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.048219    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:52:35.188649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188558    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-podinfo\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188621    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-token\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188653    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-certs\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188809    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52bb0c84-0066-4c1d-be3b-58366c6c286b-istiod-ca-cert\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188842    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-credential-socket\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188868    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-data\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188900    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:35.188990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188930    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-envoy\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.188990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.188976    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-socket\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.189194 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189009    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjjvm\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-kube-api-access-wjjvm\") pod \"52bb0c84-0066-4c1d-be3b-58366c6c286b\" (UID: \"52bb0c84-0066-4c1d-be3b-58366c6c286b\") "
Sep 19 12:52:35.189194 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189114    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:35.189289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189210    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52bb0c84-0066-4c1d-be3b-58366c6c286b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:35.189289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189217    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-data" (OuterVolumeSpecName: "istio-data") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:35.189289 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189247    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:35.189446 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189419    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52bb0c84-0066-4c1d-be3b-58366c6c286b-istiod-ca-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.189521 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189454    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-credential-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.189521 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189490    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-data\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.189521 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189503    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.189521 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.189514    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-workload-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.191146 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.191122    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:52:35.191248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.191161    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-kube-api-access-wjjvm" (OuterVolumeSpecName: "kube-api-access-wjjvm") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "kube-api-access-wjjvm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:35.191348 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.191328    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-token" (OuterVolumeSpecName: "istio-token") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:35.191410 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.191340    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52bb0c84-0066-4c1d-be3b-58366c6c286b" (UID: "52bb0c84-0066-4c1d-be3b-58366c6c286b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 12:52:35.290839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.290806    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-envoy\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.290839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.290833    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjjvm\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-kube-api-access-wjjvm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.290839 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.290844    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-podinfo\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.291064 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.290852    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52bb0c84-0066-4c1d-be3b-58366c6c286b-istio-token\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:35.376011 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.375978    2571 generic.go:356] "Generic (PLEG): container finished" podID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerID="d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785" exitCode=0
Sep 19 12:52:35.376177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.376021    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"
Sep 19 12:52:35.376177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.376061    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" event={"ID":"52bb0c84-0066-4c1d-be3b-58366c6c286b","Type":"ContainerDied","Data":"d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785"}
Sep 19 12:52:35.376177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.376112    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5" event={"ID":"52bb0c84-0066-4c1d-be3b-58366c6c286b","Type":"ContainerDied","Data":"65609d2e3b57ffc699229bedc524bcb48ef0f612b3b81370a7bbc3c57db7acf0"}
Sep 19 12:52:35.376177 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.376135    2571 scope.go:117] "RemoveContainer" containerID="d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785"
Sep 19 12:52:35.386005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.385988    2571 scope.go:117] "RemoveContainer" containerID="b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"
Sep 19 12:52:35.394531 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.394513    2571 scope.go:117] "RemoveContainer" containerID="d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785"
Sep 19 12:52:35.394834 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:52:35.394818    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785\": container with ID starting with d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785 not found: ID does not exist" containerID="d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785"
Sep 19 12:52:35.394884 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.394843    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785"} err="failed to get container status \"d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785\": rpc error: code = NotFound desc = could not find container \"d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785\": container with ID starting with d89cea4062d510d9dd209e26404e92518568b2f3c39a1fe2475dbc56c7b5a785 not found: ID does not exist"
Sep 19 12:52:35.394884 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.394863    2571 scope.go:117] "RemoveContainer" containerID="b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"
Sep 19 12:52:35.395119 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:52:35.395103    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b\": container with ID starting with b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b not found: ID does not exist" containerID="b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"
Sep 19 12:52:35.395155 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.395126    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b"} err="failed to get container status \"b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b\": rpc error: code = NotFound desc = could not find container \"b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b\": container with ID starting with b2b986d37cf28d7305fb9aadad7afae0e544ae038b0c63bc58c5271ce955b85b not found: ID does not exist"
Sep 19 12:52:35.398444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.398421    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"]
Sep 19 12:52:35.405342 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:35.405320    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-513a51-openshift-default-7b4dbc5674-842h5"]
Sep 19 12:52:36.739818 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:36.739787    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" path="/var/lib/kubelet/pods/52bb0c84-0066-4c1d-be3b-58366c6c286b/volumes"
Sep 19 12:52:40.885036 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:40.884991    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:40.900089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:40.900059    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"]
Sep 19 12:52:40.903517 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:40.903495    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-2a487c-29830365-cqg5w"]
Sep 19 12:52:40.930287 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:40.930260    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"]
Sep 19 12:52:40.930570 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:40.930534    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r" podUID="9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" containerName="payload-processing" containerID="cri-o://b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb" gracePeriod=30
Sep 19 12:52:40.983923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:40.983889    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:41.084000 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.083954    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:41.152902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.152818    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:41.175411 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.175386    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:52:41.347406 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.347375    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cpjl\" (UniqueName: \"kubernetes.io/projected/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-kube-api-access-2cpjl\") pod \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") "
Sep 19 12:52:41.347634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.347420    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-plugins-config-volume\") pod \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\" (UID: \"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737\") "
Sep 19 12:52:41.347832 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.347807    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" (UID: "9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:41.349525 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.349502    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-kube-api-access-2cpjl" (OuterVolumeSpecName: "kube-api-access-2cpjl") pod "9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" (UID: "9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737"). InnerVolumeSpecName "kube-api-access-2cpjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:41.409350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.409257    2571 generic.go:356] "Generic (PLEG): container finished" podID="9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" containerID="b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb" exitCode=0
Sep 19 12:52:41.409350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.409302    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"
Sep 19 12:52:41.409622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.409348    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r" event={"ID":"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737","Type":"ContainerDied","Data":"b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb"}
Sep 19 12:52:41.409622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.409409    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r" event={"ID":"9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737","Type":"ContainerDied","Data":"557ff9fe24ba09dfaf82360a9d619de97bb878e2f05b6282f59fc375fcfb019b"}
Sep 19 12:52:41.409622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.409437    2571 scope.go:117] "RemoveContainer" containerID="b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb"
Sep 19 12:52:41.418735 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.418719    2571 scope.go:117] "RemoveContainer" containerID="b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb"
Sep 19 12:52:41.419009 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:52:41.418990    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb\": container with ID starting with b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb not found: ID does not exist" containerID="b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb"
Sep 19 12:52:41.419079 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.419018    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb"} err="failed to get container status \"b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb\": rpc error: code = NotFound desc = could not find container \"b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb\": container with ID starting with b13e14a00fe1ceb055a84aa423a07a99be039916c759f3bac27fa708528c0edb not found: ID does not exist"
Sep 19 12:52:41.434621 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.434600    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"]
Sep 19 12:52:41.442665 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.442645    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-2a487c-5bbffc5447-6756r"]
Sep 19 12:52:41.448043 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.448028    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cpjl\" (UniqueName: \"kubernetes.io/projected/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-kube-api-access-2cpjl\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:41.448107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.448046    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:41.476376 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.476349    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:41.574297 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:41.574260    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:42.404100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:42.404064    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:42.741640 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:42.741605    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" path="/var/lib/kubelet/pods/9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737/volumes"
Sep 19 12:52:42.742009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:42.741988    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e0b2af-545c-4d22-9b66-0812d1d633ff" path="/var/lib/kubelet/pods/d9e0b2af-545c-4d22-9b66-0812d1d633ff/volumes"
Sep 19 12:52:43.349516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:43.349454    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:44.077077 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:44.077045    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:44.755829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:44.755797    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:47.685137 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:47.685104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:47.758053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:47.758016    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:47.953252 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:47.953221    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:48.447090 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:48.446998    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:49.474393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:49.474357    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:50.303850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:50.303812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:51.031369 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:51.031336    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:51.878991 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:51.878961    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:52.570257 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:52.570224    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:56.531290 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:56.531249    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:52:58.962478 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:58.962424    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:58.989783 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:58.989757    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"]
Sep 19 12:52:58.990010 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:58.989982    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c" podUID="6021844b-979f-42bf-b291-e21a7568aad8" containerName="payload-processing" containerID="cri-o://32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7" gracePeriod=30
Sep 19 12:52:58.997790 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:58.997766    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"]
Sep 19 12:52:58.997965 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:58.997947    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756" podUID="786a0536-42fe-430f-a65e-68e76502c588" containerName="payload-pre-processing" containerID="cri-o://ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57" gracePeriod=30
Sep 19 12:52:59.042247 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.042220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:59.144639 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.144586    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:59.243154 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.243066    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:59.266016 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.265993    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:52:59.268841 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.268823    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:52:59.298885 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.298863    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6021844b-979f-42bf-b291-e21a7568aad8-plugins-config-volume\") pod \"6021844b-979f-42bf-b291-e21a7568aad8\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") "
Sep 19 12:52:59.298990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.298913    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hb9lb\" (UniqueName: \"kubernetes.io/projected/6021844b-979f-42bf-b291-e21a7568aad8-kube-api-access-hb9lb\") pod \"6021844b-979f-42bf-b291-e21a7568aad8\" (UID: \"6021844b-979f-42bf-b291-e21a7568aad8\") "
Sep 19 12:52:59.298990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.298932    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfsvx\" (UniqueName: \"kubernetes.io/projected/786a0536-42fe-430f-a65e-68e76502c588-kube-api-access-mfsvx\") pod \"786a0536-42fe-430f-a65e-68e76502c588\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") "
Sep 19 12:52:59.298990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.298953    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/786a0536-42fe-430f-a65e-68e76502c588-plugins-config-volume\") pod \"786a0536-42fe-430f-a65e-68e76502c588\" (UID: \"786a0536-42fe-430f-a65e-68e76502c588\") "
Sep 19 12:52:59.299334 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.299303    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6021844b-979f-42bf-b291-e21a7568aad8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6021844b-979f-42bf-b291-e21a7568aad8" (UID: "6021844b-979f-42bf-b291-e21a7568aad8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:59.299334 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.299325    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/786a0536-42fe-430f-a65e-68e76502c588-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "786a0536-42fe-430f-a65e-68e76502c588" (UID: "786a0536-42fe-430f-a65e-68e76502c588"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:52:59.301180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.301153    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6021844b-979f-42bf-b291-e21a7568aad8-kube-api-access-hb9lb" (OuterVolumeSpecName: "kube-api-access-hb9lb") pod "6021844b-979f-42bf-b291-e21a7568aad8" (UID: "6021844b-979f-42bf-b291-e21a7568aad8"). InnerVolumeSpecName "kube-api-access-hb9lb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:59.301271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.301252    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/786a0536-42fe-430f-a65e-68e76502c588-kube-api-access-mfsvx" (OuterVolumeSpecName: "kube-api-access-mfsvx") pod "786a0536-42fe-430f-a65e-68e76502c588" (UID: "786a0536-42fe-430f-a65e-68e76502c588"). InnerVolumeSpecName "kube-api-access-mfsvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:52:59.400183 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.400158    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hb9lb\" (UniqueName: \"kubernetes.io/projected/6021844b-979f-42bf-b291-e21a7568aad8-kube-api-access-hb9lb\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:59.400183 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.400185    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfsvx\" (UniqueName: \"kubernetes.io/projected/786a0536-42fe-430f-a65e-68e76502c588-kube-api-access-mfsvx\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:59.400345 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.400195    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/786a0536-42fe-430f-a65e-68e76502c588-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:59.400345 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.400204    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6021844b-979f-42bf-b291-e21a7568aad8-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:52:59.498624 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.498545    2571 generic.go:356] "Generic (PLEG): container finished" podID="786a0536-42fe-430f-a65e-68e76502c588" containerID="ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57" exitCode=0
Sep 19 12:52:59.498624 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.498598    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"
Sep 19 12:52:59.498795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.498632    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756" event={"ID":"786a0536-42fe-430f-a65e-68e76502c588","Type":"ContainerDied","Data":"ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57"}
Sep 19 12:52:59.498795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.498683    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756" event={"ID":"786a0536-42fe-430f-a65e-68e76502c588","Type":"ContainerDied","Data":"59395a50aab6e5af58e971194ebba5c1f6dbb0a4dfa0a8d0f74ece270a053ff9"}
Sep 19 12:52:59.498795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.498703    2571 scope.go:117] "RemoveContainer" containerID="ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57"
Sep 19 12:52:59.500001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.499979    2571 generic.go:356] "Generic (PLEG): container finished" podID="6021844b-979f-42bf-b291-e21a7568aad8" containerID="32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7" exitCode=0
Sep 19 12:52:59.500107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.500014    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c" event={"ID":"6021844b-979f-42bf-b291-e21a7568aad8","Type":"ContainerDied","Data":"32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7"}
Sep 19 12:52:59.500107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.500053    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c" event={"ID":"6021844b-979f-42bf-b291-e21a7568aad8","Type":"ContainerDied","Data":"ddd92c22d0411d72eeee5a02126fc976b9779853105c4b8d3eb5e26362938a16"}
Sep 19 12:52:59.500107 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.500016    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"
Sep 19 12:52:59.510421 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.510401    2571 scope.go:117] "RemoveContainer" containerID="ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57"
Sep 19 12:52:59.510792 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:52:59.510768    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57\": container with ID starting with ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57 not found: ID does not exist" containerID="ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57"
Sep 19 12:52:59.510869 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.510803    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57"} err="failed to get container status \"ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57\": rpc error: code = NotFound desc = could not find container \"ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57\": container with ID starting with ebdacd39a44d303e7bcdcbd3f0bf147bed9f3d2b7086bae48b1c2610891aee57 not found: ID does not exist"
Sep 19 12:52:59.510869 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.510828    2571 scope.go:117] "RemoveContainer" containerID="32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7"
Sep 19 12:52:59.518356 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.518340    2571 scope.go:117] "RemoveContainer" containerID="32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7"
Sep 19 12:52:59.518605 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:52:59.518585    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7\": container with ID starting with 32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7 not found: ID does not exist" containerID="32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7"
Sep 19 12:52:59.518650 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.518614    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7"} err="failed to get container status \"32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7\": rpc error: code = NotFound desc = could not find container \"32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7\": container with ID starting with 32bb351517b5500ccf1ff7e6e048ece901d991aa11a7588c7482411e033f44b7 not found: ID does not exist"
Sep 19 12:52:59.522035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.522015    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"]
Sep 19 12:52:59.526632 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.526612    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-5068d9-68774ff6b-vf756"]
Sep 19 12:52:59.532862 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.532835    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:52:59.537982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.537959    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"]
Sep 19 12:52:59.542443 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.542419    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-5068d9-6dcb76fd5f-zjd7c"]
Sep 19 12:52:59.654622 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:52:59.654592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:00.366343 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:00.366313    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:00.739412 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:00.739378    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6021844b-979f-42bf-b291-e21a7568aad8" path="/var/lib/kubelet/pods/6021844b-979f-42bf-b291-e21a7568aad8/volumes"
Sep 19 12:53:00.739770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:00.739757    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="786a0536-42fe-430f-a65e-68e76502c588" path="/var/lib/kubelet/pods/786a0536-42fe-430f-a65e-68e76502c588/volumes"
Sep 19 12:53:01.327765 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:01.327731    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:02.188574 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:02.188537    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:05.421426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:05.421391    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:05.521876 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:05.521844    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:05.677848 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:05.677755    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:07.179630 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:07.179590    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:12.695735 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:12.695698    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:13.947617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:13.947576    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:18.626001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:18.625968    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:53:18.928960 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:18.928883    2571 scope.go:117] "RemoveContainer" containerID="9b29bc48c768158a9498ade9d1c9b3c496ba6b1275e2f8b0a94964e9222e8638"
Sep 19 12:53:23.749588 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749556    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"]
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749884    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" containerName="payload-processing"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749895    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" containerName="payload-processing"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749905    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749911    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749921    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a47b9066-4012-4ba0-84a4-e37832db3adf" containerName="main"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749926    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="a47b9066-4012-4ba0-84a4-e37832db3adf" containerName="main"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749934    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7533bbc7-efa7-438f-843f-9ea8142e8701" containerName="main"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749939    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7533bbc7-efa7-438f-843f-9ea8142e8701" containerName="main"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749947    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7c386f9-b795-4dcb-a600-d38dcf5af343" containerName="main"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749952    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7c386f9-b795-4dcb-a600-d38dcf5af343" containerName="main"
Sep 19 12:53:23.749961 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749964    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e5fd8e0-4321-4508-98ca-5bcd2769b322" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749969    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e5fd8e0-4321-4508-98ca-5bcd2769b322" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749976    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51fe2b4a-6611-42dd-95a5-f4834e5a7caa" containerName="main"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749980    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="51fe2b4a-6611-42dd-95a5-f4834e5a7caa" containerName="main"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749988    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="786a0536-42fe-430f-a65e-68e76502c588" containerName="payload-pre-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749993    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="786a0536-42fe-430f-a65e-68e76502c588" containerName="payload-pre-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.749998    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6021844b-979f-42bf-b291-e21a7568aad8" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750003    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="6021844b-979f-42bf-b291-e21a7568aad8" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750053    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d2a54cc-5e0a-4cdf-9ac5-dc0a38c1c737" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750060    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7533bbc7-efa7-438f-843f-9ea8142e8701" containerName="main"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750065    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750070    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="6021844b-979f-42bf-b291-e21a7568aad8" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750077    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="786a0536-42fe-430f-a65e-68e76502c588" containerName="payload-pre-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750082    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e5fd8e0-4321-4508-98ca-5bcd2769b322" containerName="payload-processing"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750088    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="51fe2b4a-6611-42dd-95a5-f4834e5a7caa" containerName="main"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750092    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="a47b9066-4012-4ba0-84a4-e37832db3adf" containerName="main"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750099    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7c386f9-b795-4dcb-a600-d38dcf5af343" containerName="main"
Sep 19 12:53:23.750344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.750105    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy"
Sep 19 12:53:23.754540 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.754522    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:23.757394 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.757362    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"openshift-service-ca.crt\""
Sep 19 12:53:23.757540 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.757366    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"kube-root-ca.crt\""
Sep 19 12:53:23.758503 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.758482    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"sub-test-model-b0e30e-kserve-self-signed-certs\""
Sep 19 12:53:23.758619 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.758508    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-b0e30e\"/\"default-dockercfg-gmwjx\""
Sep 19 12:53:23.763122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.763097    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"]
Sep 19 12:53:23.776490 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.776445    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:23.930082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.930054    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qvws\" (UniqueName: \"kubernetes.io/projected/77a94435-dabb-44e3-b2cf-8840c15e63c0-kube-api-access-5qvws\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:23.930082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.930086    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-tmp-dir\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:23.930293 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.930122    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a94435-dabb-44e3-b2cf-8840c15e63c0-tls-certs\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:23.930293 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.930152    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-model-cache\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:23.930293 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.930207    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-dshm\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:23.930293 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:23.930280    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-home\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.031598 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031526    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qvws\" (UniqueName: \"kubernetes.io/projected/77a94435-dabb-44e3-b2cf-8840c15e63c0-kube-api-access-5qvws\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.031598 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031561    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-tmp-dir\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.031598 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031590    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a94435-dabb-44e3-b2cf-8840c15e63c0-tls-certs\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.031850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031624    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-model-cache\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.031850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031678    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-dshm\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.031850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031721    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-home\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.032021 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.031940    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-tmp-dir\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.032075 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.032016    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-model-cache\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.032119 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.032080    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-home\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.033922 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.033903    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-dshm\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.034029 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.034014    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a94435-dabb-44e3-b2cf-8840c15e63c0-tls-certs\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.042569 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.042551    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qvws\" (UniqueName: \"kubernetes.io/projected/77a94435-dabb-44e3-b2cf-8840c15e63c0-kube-api-access-5qvws\") pod \"sub-test-model-b0e30e-kserve-8459459d89-r5x29\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") " pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.065045 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.065029    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:24.335241 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.335217    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"]
Sep 19 12:53:24.336843 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:53:24.336820    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77a94435_dabb_44e3_b2cf_8840c15e63c0.slice/crio-d62cd054620765c8403ec00638e2af65e4ae8cffd0be127f7f08a3b7b05e312f WatchSource:0}: Error finding container d62cd054620765c8403ec00638e2af65e4ae8cffd0be127f7f08a3b7b05e312f: Status 404 returned error can't find the container with id d62cd054620765c8403ec00638e2af65e4ae8cffd0be127f7f08a3b7b05e312f
Sep 19 12:53:24.380352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.380318    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:24.621546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.621444    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29" event={"ID":"77a94435-dabb-44e3-b2cf-8840c15e63c0","Type":"ContainerStarted","Data":"4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af"}
Sep 19 12:53:24.621546 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.621516    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29" event={"ID":"77a94435-dabb-44e3-b2cf-8840c15e63c0","Type":"ContainerStarted","Data":"d62cd054620765c8403ec00638e2af65e4ae8cffd0be127f7f08a3b7b05e312f"}
Sep 19 12:53:24.642823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.642774    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29" podStartSLOduration=1.6427583270000001 podStartE2EDuration="1.642758327s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:53:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:53:24.641046647 +0000 UTC m=+1688.001289590" watchObservedRunningTime="2026-09-19 12:53:24.642758327 +0000 UTC m=+1688.003001284"
Sep 19 12:53:24.869145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:24.869113    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:25.973795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:25.973760    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:26.676713 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:26.676677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:27.386730 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:27.386684    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:28.365524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:28.365477    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:29.364228 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:29.364185    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"]
Sep 19 12:53:29.375485 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:29.375447    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"]
Sep 19 12:53:29.404706 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:29.404675    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"]
Sep 19 12:53:29.495350 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:29.495314    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:34.065422 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:34.065333    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:34.065907 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:34.065496    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:34.078111 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:34.078086    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:34.681341 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:34.681310    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:53:40.120651 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.120613    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"]
Sep 19 12:53:40.121115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.120899    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94" podUID="39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" containerName="maas-api" containerID="cri-o://f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf" gracePeriod=30
Sep 19 12:53:40.155977 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.155951    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:40.238188 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.238005    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:40.317892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.317850    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:40.364057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.364034    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:53:40.422687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.422595    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:40.476278 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.476244    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-metrics-certs\") pod \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") "
Sep 19 12:53:40.476504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.476298    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkhp9\" (UniqueName: \"kubernetes.io/projected/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-kube-api-access-qkhp9\") pod \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") "
Sep 19 12:53:40.476504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.476371    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-openshift-service-ca\") pod \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") "
Sep 19 12:53:40.476504 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.476417    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-maas-api-tls\") pod \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\" (UID: \"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c\") "
Sep 19 12:53:40.476765 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.476737    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" (UID: "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:53:40.478547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.478522    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-kube-api-access-qkhp9" (OuterVolumeSpecName: "kube-api-access-qkhp9") pod "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" (UID: "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c"). InnerVolumeSpecName "kube-api-access-qkhp9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:40.478547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.478536    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" (UID: "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:53:40.478547 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.478520    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" (UID: "39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:53:40.505662 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.505630    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:40.553665 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.553631    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:53:40.577821 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.577792    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:40.577928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.577823    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:40.577928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.577837    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:40.577928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.577849    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkhp9\" (UniqueName: \"kubernetes.io/projected/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c-kube-api-access-qkhp9\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:40.698656 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.698559    2571 generic.go:356] "Generic (PLEG): container finished" podID="39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" containerID="f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf" exitCode=0
Sep 19 12:53:40.698656 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.698614    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"
Sep 19 12:53:40.698828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.698650    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94" event={"ID":"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c","Type":"ContainerDied","Data":"f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf"}
Sep 19 12:53:40.698828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.698703    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94" event={"ID":"39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c","Type":"ContainerDied","Data":"bea0d8572737302a30399cea0d36c06a511f83fe5139496d70474b3ded420e8c"}
Sep 19 12:53:40.698828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.698725    2571 scope.go:117] "RemoveContainer" containerID="f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf"
Sep 19 12:53:40.707800 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.707785    2571 scope.go:117] "RemoveContainer" containerID="f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf"
Sep 19 12:53:40.708067 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:53:40.708049    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf\": container with ID starting with f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf not found: ID does not exist" containerID="f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf"
Sep 19 12:53:40.708147 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.708074    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf"} err="failed to get container status \"f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf\": rpc error: code = NotFound desc = could not find container \"f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf\": container with ID starting with f69b21cbc320bb574fcac76408180a086bac78faa4d48d4c35858cccc98015cf not found: ID does not exist"
Sep 19 12:53:40.741931 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.741905    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"]
Sep 19 12:53:40.752940 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.752919    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-04a668-6f796d4f4b-5kp94"]
Sep 19 12:53:40.860631 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.860598    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:40.938999 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:40.938966    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:41.338040 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:41.338007    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:41.792795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:41.792762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:42.599816 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:42.599780    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:42.739848 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:42.739819    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" path="/var/lib/kubelet/pods/39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c/volumes"
Sep 19 12:53:44.383403 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.383362    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9" podUID="df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" containerName="main" containerID="cri-o://c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9" gracePeriod=45
Sep 19 12:53:44.395892 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.395857    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh" podUID="7906c6bf-e287-43d6-9463-993052a58779" containerName="main" containerID="cri-o://fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14" gracePeriod=45
Sep 19 12:53:44.427100 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.427064    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl" podUID="9076e402-0fa9-4f76-882a-f7a927445602" containerName="main" containerID="cri-o://ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b" gracePeriod=45
Sep 19 12:53:44.701195 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.701175    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:53:44.704056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.704037    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:53:44.706976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.706959    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:53:44.717716 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.717691    2571 generic.go:356] "Generic (PLEG): container finished" podID="9076e402-0fa9-4f76-882a-f7a927445602" containerID="ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b" exitCode=0
Sep 19 12:53:44.717809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.717721    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"
Sep 19 12:53:44.717809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.717742    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl" event={"ID":"9076e402-0fa9-4f76-882a-f7a927445602","Type":"ContainerDied","Data":"ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b"}
Sep 19 12:53:44.717809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.717781    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl" event={"ID":"9076e402-0fa9-4f76-882a-f7a927445602","Type":"ContainerDied","Data":"8763dbf191d2ede4c45c778f04d983fefc4149d6648c7f99e10bfaade5a619c7"}
Sep 19 12:53:44.717809 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.717807    2571 scope.go:117] "RemoveContainer" containerID="ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b"
Sep 19 12:53:44.719145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.719125    2571 generic.go:356] "Generic (PLEG): container finished" podID="df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" containerID="c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9" exitCode=0
Sep 19 12:53:44.719226 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.719170    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"
Sep 19 12:53:44.719226 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.719205    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9" event={"ID":"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8","Type":"ContainerDied","Data":"c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9"}
Sep 19 12:53:44.719316 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.719237    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9" event={"ID":"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8","Type":"ContainerDied","Data":"474a7d5e650af95a3b9e7fc3605a2b13c5fd1434ffce25dc2ed59707c80bb7d8"}
Sep 19 12:53:44.720471 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.720440    2571 generic.go:356] "Generic (PLEG): container finished" podID="7906c6bf-e287-43d6-9463-993052a58779" containerID="fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14" exitCode=0
Sep 19 12:53:44.720566 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.720483    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"
Sep 19 12:53:44.720566 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.720498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh" event={"ID":"7906c6bf-e287-43d6-9463-993052a58779","Type":"ContainerDied","Data":"fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14"}
Sep 19 12:53:44.720566 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.720525    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh" event={"ID":"7906c6bf-e287-43d6-9463-993052a58779","Type":"ContainerDied","Data":"8d5e390308e2e35341b5ce46372ecf0f95303c97da3974107e6777b75b7b8992"}
Sep 19 12:53:44.725974 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.725943    2571 scope.go:117] "RemoveContainer" containerID="ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b"
Sep 19 12:53:44.726323 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:53:44.726292    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b\": container with ID starting with ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b not found: ID does not exist" containerID="ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b"
Sep 19 12:53:44.726413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.726331    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b"} err="failed to get container status \"ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b\": rpc error: code = NotFound desc = could not find container \"ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b\": container with ID starting with ea39ea985f94ca6a09bb3906e58ff252d5f5cbcbd29367e2b9ff911b64c54f4b not found: ID does not exist"
Sep 19 12:53:44.726413 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.726348    2571 scope.go:117] "RemoveContainer" containerID="c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9"
Sep 19 12:53:44.736646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.736630    2571 scope.go:117] "RemoveContainer" containerID="c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9"
Sep 19 12:53:44.736935 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:53:44.736917    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9\": container with ID starting with c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9 not found: ID does not exist" containerID="c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9"
Sep 19 12:53:44.737001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.736941    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9"} err="failed to get container status \"c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9\": rpc error: code = NotFound desc = could not find container \"c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9\": container with ID starting with c50164af68d7d0b064fbf5c7d019c8a93ab799460fa04f183c0bcf609c8711c9 not found: ID does not exist"
Sep 19 12:53:44.737001 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.736956    2571 scope.go:117] "RemoveContainer" containerID="fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14"
Sep 19 12:53:44.747094 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.747074    2571 scope.go:117] "RemoveContainer" containerID="fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14"
Sep 19 12:53:44.747373 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:53:44.747354    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14\": container with ID starting with fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14 not found: ID does not exist" containerID="fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14"
Sep 19 12:53:44.747447 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.747384    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14"} err="failed to get container status \"fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14\": rpc error: code = NotFound desc = could not find container \"fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14\": container with ID starting with fb99b956b5243ee8429710f4e9703a5b9531ba38fc465b9c3274fe80948b7b14 not found: ID does not exist"
Sep 19 12:53:44.818231 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818196    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-home\") pod \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818239    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-model-cache\") pod \"9076e402-0fa9-4f76-882a-f7a927445602\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818269    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpjr7\" (UniqueName: \"kubernetes.io/projected/7906c6bf-e287-43d6-9463-993052a58779-kube-api-access-qpjr7\") pod \"7906c6bf-e287-43d6-9463-993052a58779\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818298    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-tmp-dir\") pod \"9076e402-0fa9-4f76-882a-f7a927445602\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818320    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9076e402-0fa9-4f76-882a-f7a927445602-tls-certs\") pod \"9076e402-0fa9-4f76-882a-f7a927445602\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818341    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-model-cache\") pod \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818366    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pvfw\" (UniqueName: \"kubernetes.io/projected/9076e402-0fa9-4f76-882a-f7a927445602-kube-api-access-2pvfw\") pod \"9076e402-0fa9-4f76-882a-f7a927445602\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") "
Sep 19 12:53:44.818418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818394    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7906c6bf-e287-43d6-9463-993052a58779-tls-certs\") pod \"7906c6bf-e287-43d6-9463-993052a58779\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818425    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-dshm\") pod \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818448    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bl8m4\" (UniqueName: \"kubernetes.io/projected/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-kube-api-access-bl8m4\") pod \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818542    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-home" (OuterVolumeSpecName: "home") pod "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" (UID: "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818558    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9076e402-0fa9-4f76-882a-f7a927445602" (UID: "9076e402-0fa9-4f76-882a-f7a927445602"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818552    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-tmp-dir\") pod \"7906c6bf-e287-43d6-9463-993052a58779\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818591    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-model-cache" (OuterVolumeSpecName: "model-cache") pod "9076e402-0fa9-4f76-882a-f7a927445602" (UID: "9076e402-0fa9-4f76-882a-f7a927445602"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818623    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tmp-dir\") pod \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818654    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tls-certs\") pod \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\" (UID: \"df74ed2f-6bcb-4c47-aea9-c81ab19b05a8\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818691    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-model-cache\") pod \"7906c6bf-e287-43d6-9463-993052a58779\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818728    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-home\") pod \"9076e402-0fa9-4f76-882a-f7a927445602\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818763    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7906c6bf-e287-43d6-9463-993052a58779" (UID: "7906c6bf-e287-43d6-9463-993052a58779"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818771    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-model-cache" (OuterVolumeSpecName: "model-cache") pod "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" (UID: "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818798    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-dshm\") pod \"9076e402-0fa9-4f76-882a-f7a927445602\" (UID: \"9076e402-0fa9-4f76-882a-f7a927445602\") "
Sep 19 12:53:44.818829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818826    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-home\") pod \"7906c6bf-e287-43d6-9463-993052a58779\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") "
Sep 19 12:53:44.819613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.818848    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-dshm\") pod \"7906c6bf-e287-43d6-9463-993052a58779\" (UID: \"7906c6bf-e287-43d6-9463-993052a58779\") "
Sep 19 12:53:44.819613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.819186    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.819613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.819203    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.819613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.819217    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.819613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.819230    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.819613 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.819245    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.819923 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.819869    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-home" (OuterVolumeSpecName: "home") pod "9076e402-0fa9-4f76-882a-f7a927445602" (UID: "9076e402-0fa9-4f76-882a-f7a927445602"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.820097 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.820072    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" (UID: "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.820174 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.820087    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-model-cache" (OuterVolumeSpecName: "model-cache") pod "7906c6bf-e287-43d6-9463-993052a58779" (UID: "7906c6bf-e287-43d6-9463-993052a58779"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.820174 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.820162    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-home" (OuterVolumeSpecName: "home") pod "7906c6bf-e287-43d6-9463-993052a58779" (UID: "7906c6bf-e287-43d6-9463-993052a58779"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.821003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.820965    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7906c6bf-e287-43d6-9463-993052a58779-kube-api-access-qpjr7" (OuterVolumeSpecName: "kube-api-access-qpjr7") pod "7906c6bf-e287-43d6-9463-993052a58779" (UID: "7906c6bf-e287-43d6-9463-993052a58779"). InnerVolumeSpecName "kube-api-access-qpjr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:44.821266 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.821241    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-kube-api-access-bl8m4" (OuterVolumeSpecName: "kube-api-access-bl8m4") pod "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" (UID: "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8"). InnerVolumeSpecName "kube-api-access-bl8m4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:44.821744 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.821712    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9076e402-0fa9-4f76-882a-f7a927445602-kube-api-access-2pvfw" (OuterVolumeSpecName: "kube-api-access-2pvfw") pod "9076e402-0fa9-4f76-882a-f7a927445602" (UID: "9076e402-0fa9-4f76-882a-f7a927445602"). InnerVolumeSpecName "kube-api-access-2pvfw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:53:44.821744 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.821716    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-dshm" (OuterVolumeSpecName: "dshm") pod "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" (UID: "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.821886 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.821758    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9076e402-0fa9-4f76-882a-f7a927445602-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9076e402-0fa9-4f76-882a-f7a927445602" (UID: "9076e402-0fa9-4f76-882a-f7a927445602"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:53:44.821886 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.821769    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" (UID: "df74ed2f-6bcb-4c47-aea9-c81ab19b05a8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:53:44.821992 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.821975    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-dshm" (OuterVolumeSpecName: "dshm") pod "7906c6bf-e287-43d6-9463-993052a58779" (UID: "7906c6bf-e287-43d6-9463-993052a58779"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.822291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.822268    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7906c6bf-e287-43d6-9463-993052a58779-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7906c6bf-e287-43d6-9463-993052a58779" (UID: "7906c6bf-e287-43d6-9463-993052a58779"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:53:44.822394 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.822327    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-dshm" (OuterVolumeSpecName: "dshm") pod "9076e402-0fa9-4f76-882a-f7a927445602" (UID: "9076e402-0fa9-4f76-882a-f7a927445602"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:53:44.920727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920692    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpjr7\" (UniqueName: \"kubernetes.io/projected/7906c6bf-e287-43d6-9463-993052a58779-kube-api-access-qpjr7\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920720    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9076e402-0fa9-4f76-882a-f7a927445602-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920730    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pvfw\" (UniqueName: \"kubernetes.io/projected/9076e402-0fa9-4f76-882a-f7a927445602-kube-api-access-2pvfw\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920738    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7906c6bf-e287-43d6-9463-993052a58779-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920747    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920755    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bl8m4\" (UniqueName: \"kubernetes.io/projected/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-kube-api-access-bl8m4\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920763    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920771    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920779    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920787    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920794    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9076e402-0fa9-4f76-882a-f7a927445602-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920801    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:44.920945 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:44.920808    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7906c6bf-e287-43d6-9463-993052a58779-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:53:45.050700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:45.050669    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"]
Sep 19 12:53:45.055801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:45.055775    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-2-w4-04a668-kserve-8c675db8f-zsbt9"]
Sep 19 12:53:45.068074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:45.068049    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"]
Sep 19 12:53:45.075378 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:45.075354    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/simulator-w4-04a668-kserve-85499c78b6-fhnpl"]
Sep 19 12:53:45.086871 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:45.086848    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"]
Sep 19 12:53:45.090927 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:45.090902    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-04a668/distinct-w4-04a668-kserve-5f495dd454-t25xh"]
Sep 19 12:53:46.733859 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:46.733822    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:46.739524 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:46.739498    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7906c6bf-e287-43d6-9463-993052a58779" path="/var/lib/kubelet/pods/7906c6bf-e287-43d6-9463-993052a58779/volumes"
Sep 19 12:53:46.739875 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:46.739861    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9076e402-0fa9-4f76-882a-f7a927445602" path="/var/lib/kubelet/pods/9076e402-0fa9-4f76-882a-f7a927445602/volumes"
Sep 19 12:53:46.740208 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:46.740197    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" path="/var/lib/kubelet/pods/df74ed2f-6bcb-4c47-aea9-c81ab19b05a8/volumes"
Sep 19 12:53:46.847560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:46.847523    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:46.983129 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:46.983098    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:47.191530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:47.191427    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:53:53.629068 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:53:53.629035    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:54:06.321198 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:06.321163    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:06.419155 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:06.419118    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:06.937228 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:06.937194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:13.735114 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735085    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"]
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735368    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" containerName="maas-api"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735380    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" containerName="maas-api"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735387    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7906c6bf-e287-43d6-9463-993052a58779" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735392    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="7906c6bf-e287-43d6-9463-993052a58779" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735402    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735408    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="52bb0c84-0066-4c1d-be3b-58366c6c286b" containerName="istio-proxy"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735423    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9076e402-0fa9-4f76-882a-f7a927445602" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735428    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9076e402-0fa9-4f76-882a-f7a927445602" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735435    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735440    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735504    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a1d15a-79d8-40b5-b449-9fa6b4a4ca2c" containerName="maas-api"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735512    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="7906c6bf-e287-43d6-9463-993052a58779" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735518    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9076e402-0fa9-4f76-882a-f7a927445602" containerName="main"
Sep 19 12:54:13.735612 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.735523    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="df74ed2f-6bcb-4c47-aea9-c81ab19b05a8" containerName="main"
Sep 19 12:54:13.738314 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.738293    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.741037 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.741016    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"default-dockercfg-2cfx6\""
Sep 19 12:54:13.742339 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.742318    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"kube-root-ca.crt\""
Sep 19 12:54:13.742444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.742318    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"sub-test-model-5d6124-kserve-self-signed-certs\""
Sep 19 12:54:13.742444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.742348    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"openshift-service-ca.crt\""
Sep 19 12:54:13.748275 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.748250    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"]
Sep 19 12:54:13.765963 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.765935    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:13.875121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.875088    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-dshm\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.875335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.875132    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-tmp-dir\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.875335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.875154    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/556e2861-dc40-424c-b394-34f1a23d01f3-tls-certs\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.875335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.875216    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ql47n\" (UniqueName: \"kubernetes.io/projected/556e2861-dc40-424c-b394-34f1a23d01f3-kube-api-access-ql47n\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.875335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.875259    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-home\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.875335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.875301    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-model-cache\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976058 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976020    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-dshm\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976058 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976060    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-tmp-dir\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976089    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/556e2861-dc40-424c-b394-34f1a23d01f3-tls-certs\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976111    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ql47n\" (UniqueName: \"kubernetes.io/projected/556e2861-dc40-424c-b394-34f1a23d01f3-kube-api-access-ql47n\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976133    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-home\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976318 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976161    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-model-cache\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976564 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976548    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-tmp-dir\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976628 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976606    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-model-cache\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.976672 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.976645    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-home\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.978382 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.978361    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-dshm\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.978557 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.978539    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/556e2861-dc40-424c-b394-34f1a23d01f3-tls-certs\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:13.987429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:13.987379    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ql47n\" (UniqueName: \"kubernetes.io/projected/556e2861-dc40-424c-b394-34f1a23d01f3-kube-api-access-ql47n\") pod \"sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:14.048673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:14.048637    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:14.323717 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:14.323690    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"]
Sep 19 12:54:14.325153 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:54:14.325128    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod556e2861_dc40_424c_b394_34f1a23d01f3.slice/crio-682ba6b403ab2aa9a85be2a7658220ae194aa1b6f00b885f59c18bf6c08d3e5b WatchSource:0}: Error finding container 682ba6b403ab2aa9a85be2a7658220ae194aa1b6f00b885f59c18bf6c08d3e5b: Status 404 returned error can't find the container with id 682ba6b403ab2aa9a85be2a7658220ae194aa1b6f00b885f59c18bf6c08d3e5b
Sep 19 12:54:14.862325 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:14.862284    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb" event={"ID":"556e2861-dc40-424c-b394-34f1a23d01f3","Type":"ContainerStarted","Data":"b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b"}
Sep 19 12:54:14.862752 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:14.862334    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb" event={"ID":"556e2861-dc40-424c-b394-34f1a23d01f3","Type":"ContainerStarted","Data":"682ba6b403ab2aa9a85be2a7658220ae194aa1b6f00b885f59c18bf6c08d3e5b"}
Sep 19 12:54:14.880455 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:14.880396    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb" podStartSLOduration=1.880378093 podStartE2EDuration="1.880378093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:54:14.878754487 +0000 UTC m=+1738.238997429" watchObservedRunningTime="2026-09-19 12:54:14.880378093 +0000 UTC m=+1738.240621035"
Sep 19 12:54:17.228387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:17.228350    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:54:21.252298 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:21.252260    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:24.049240 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:24.049208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:24.049687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:24.049279    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:24.061826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:24.061804    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:24.919260 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:24.919232    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:54:32.831128 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:32.831096    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:54:42.933407 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:42.933367    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:54:52.750365 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:52.750333    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"]
Sep 19 12:54:52.764780 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:52.764753    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:53.808284 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:53.808252    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"]
Sep 19 12:54:53.831934 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:53.831906    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:54:56.452456 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:56.452421    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:54:59.360698 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:54:59.360665    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:55:07.769078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:07.768980    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29" podUID="77a94435-dabb-44e3-b2cf-8840c15e63c0" containerName="main" containerID="cri-o://4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af" gracePeriod=45
Sep 19 12:55:08.002617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.002594    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:55:08.039538 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039455    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qvws\" (UniqueName: \"kubernetes.io/projected/77a94435-dabb-44e3-b2cf-8840c15e63c0-kube-api-access-5qvws\") pod \"77a94435-dabb-44e3-b2cf-8840c15e63c0\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") "
Sep 19 12:55:08.039538 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039500    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-tmp-dir\") pod \"77a94435-dabb-44e3-b2cf-8840c15e63c0\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") "
Sep 19 12:55:08.039538 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039533    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-home\") pod \"77a94435-dabb-44e3-b2cf-8840c15e63c0\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") "
Sep 19 12:55:08.039760 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039581    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-dshm\") pod \"77a94435-dabb-44e3-b2cf-8840c15e63c0\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") "
Sep 19 12:55:08.039760 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039638    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a94435-dabb-44e3-b2cf-8840c15e63c0-tls-certs\") pod \"77a94435-dabb-44e3-b2cf-8840c15e63c0\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") "
Sep 19 12:55:08.039760 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039666    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-model-cache\") pod \"77a94435-dabb-44e3-b2cf-8840c15e63c0\" (UID: \"77a94435-dabb-44e3-b2cf-8840c15e63c0\") "
Sep 19 12:55:08.039921 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039765    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "77a94435-dabb-44e3-b2cf-8840c15e63c0" (UID: "77a94435-dabb-44e3-b2cf-8840c15e63c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:08.039921 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039800    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-home" (OuterVolumeSpecName: "home") pod "77a94435-dabb-44e3-b2cf-8840c15e63c0" (UID: "77a94435-dabb-44e3-b2cf-8840c15e63c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:08.040033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039971    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:08.040033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.039984    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:08.040033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.040004    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "77a94435-dabb-44e3-b2cf-8840c15e63c0" (UID: "77a94435-dabb-44e3-b2cf-8840c15e63c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:08.041757 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.041729    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-dshm" (OuterVolumeSpecName: "dshm") pod "77a94435-dabb-44e3-b2cf-8840c15e63c0" (UID: "77a94435-dabb-44e3-b2cf-8840c15e63c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:08.041860 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.041751    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77a94435-dabb-44e3-b2cf-8840c15e63c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "77a94435-dabb-44e3-b2cf-8840c15e63c0" (UID: "77a94435-dabb-44e3-b2cf-8840c15e63c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:55:08.041860 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.041792    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77a94435-dabb-44e3-b2cf-8840c15e63c0-kube-api-access-5qvws" (OuterVolumeSpecName: "kube-api-access-5qvws") pod "77a94435-dabb-44e3-b2cf-8840c15e63c0" (UID: "77a94435-dabb-44e3-b2cf-8840c15e63c0"). InnerVolumeSpecName "kube-api-access-5qvws". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:55:08.105646 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.105615    2571 generic.go:356] "Generic (PLEG): container finished" podID="77a94435-dabb-44e3-b2cf-8840c15e63c0" containerID="4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af" exitCode=0
Sep 19 12:55:08.105778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.105656    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"
Sep 19 12:55:08.105778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.105697    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29" event={"ID":"77a94435-dabb-44e3-b2cf-8840c15e63c0","Type":"ContainerDied","Data":"4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af"}
Sep 19 12:55:08.105778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.105744    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29" event={"ID":"77a94435-dabb-44e3-b2cf-8840c15e63c0","Type":"ContainerDied","Data":"d62cd054620765c8403ec00638e2af65e4ae8cffd0be127f7f08a3b7b05e312f"}
Sep 19 12:55:08.105778 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.105763    2571 scope.go:117] "RemoveContainer" containerID="4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af"
Sep 19 12:55:08.114344 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.114328    2571 scope.go:117] "RemoveContainer" containerID="4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af"
Sep 19 12:55:08.114626 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:55:08.114610    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af\": container with ID starting with 4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af not found: ID does not exist" containerID="4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af"
Sep 19 12:55:08.114685 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.114631    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af"} err="failed to get container status \"4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af\": rpc error: code = NotFound desc = could not find container \"4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af\": container with ID starting with 4f82c1efbf388b464793849b30eda82cbb5465bd58ed36bc27c5c74fe1d542af not found: ID does not exist"
Sep 19 12:55:08.126023 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.126003    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"]
Sep 19 12:55:08.131686 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.131669    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-b0e30e/sub-test-model-b0e30e-kserve-8459459d89-r5x29"]
Sep 19 12:55:08.140655 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.140640    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:08.140655 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.140656    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/77a94435-dabb-44e3-b2cf-8840c15e63c0-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:08.140767 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.140666    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/77a94435-dabb-44e3-b2cf-8840c15e63c0-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:08.140767 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.140675    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qvws\" (UniqueName: \"kubernetes.io/projected/77a94435-dabb-44e3-b2cf-8840c15e63c0-kube-api-access-5qvws\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:08.740061 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.740025    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77a94435-dabb-44e3-b2cf-8840c15e63c0" path="/var/lib/kubelet/pods/77a94435-dabb-44e3-b2cf-8840c15e63c0/volumes"
Sep 19 12:55:08.826770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:08.826716    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb" podUID="556e2861-dc40-424c-b394-34f1a23d01f3" containerName="main" containerID="cri-o://b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b" gracePeriod=45
Sep 19 12:55:09.023666 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.023596    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:55:09.074404 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.074384    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:55:09.111624 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.111594    2571 generic.go:356] "Generic (PLEG): container finished" podID="556e2861-dc40-424c-b394-34f1a23d01f3" containerID="b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b" exitCode=0
Sep 19 12:55:09.111801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.111635    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"
Sep 19 12:55:09.111801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.111678    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb" event={"ID":"556e2861-dc40-424c-b394-34f1a23d01f3","Type":"ContainerDied","Data":"b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b"}
Sep 19 12:55:09.111801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.111714    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb" event={"ID":"556e2861-dc40-424c-b394-34f1a23d01f3","Type":"ContainerDied","Data":"682ba6b403ab2aa9a85be2a7658220ae194aa1b6f00b885f59c18bf6c08d3e5b"}
Sep 19 12:55:09.111801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.111731    2571 scope.go:117] "RemoveContainer" containerID="b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b"
Sep 19 12:55:09.119684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.119668    2571 scope.go:117] "RemoveContainer" containerID="b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b"
Sep 19 12:55:09.119938 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:55:09.119922    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b\": container with ID starting with b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b not found: ID does not exist" containerID="b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b"
Sep 19 12:55:09.119980 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.119945    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b"} err="failed to get container status \"b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b\": rpc error: code = NotFound desc = could not find container \"b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b\": container with ID starting with b21179205cc9e16978dc42f87b113357b200f6d9792ee5a3fff864c1aca8341b not found: ID does not exist"
Sep 19 12:55:09.149409 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149385    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-tmp-dir\") pod \"556e2861-dc40-424c-b394-34f1a23d01f3\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") "
Sep 19 12:55:09.149553 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149418    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ql47n\" (UniqueName: \"kubernetes.io/projected/556e2861-dc40-424c-b394-34f1a23d01f3-kube-api-access-ql47n\") pod \"556e2861-dc40-424c-b394-34f1a23d01f3\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") "
Sep 19 12:55:09.149553 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149453    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/556e2861-dc40-424c-b394-34f1a23d01f3-tls-certs\") pod \"556e2861-dc40-424c-b394-34f1a23d01f3\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") "
Sep 19 12:55:09.149684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149663    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "556e2861-dc40-424c-b394-34f1a23d01f3" (UID: "556e2861-dc40-424c-b394-34f1a23d01f3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:09.149750 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149734    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-home\") pod \"556e2861-dc40-424c-b394-34f1a23d01f3\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") "
Sep 19 12:55:09.149812 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149781    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-model-cache\") pod \"556e2861-dc40-424c-b394-34f1a23d01f3\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") "
Sep 19 12:55:09.149870 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149826    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-dshm\") pod \"556e2861-dc40-424c-b394-34f1a23d01f3\" (UID: \"556e2861-dc40-424c-b394-34f1a23d01f3\") "
Sep 19 12:55:09.149972 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149933    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-home" (OuterVolumeSpecName: "home") pod "556e2861-dc40-424c-b394-34f1a23d01f3" (UID: "556e2861-dc40-424c-b394-34f1a23d01f3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:09.150035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.149999    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-model-cache" (OuterVolumeSpecName: "model-cache") pod "556e2861-dc40-424c-b394-34f1a23d01f3" (UID: "556e2861-dc40-424c-b394-34f1a23d01f3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:09.150140 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.150120    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:09.150220 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.150141    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:09.150220 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.150154    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:09.151770 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.151750    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/556e2861-dc40-424c-b394-34f1a23d01f3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "556e2861-dc40-424c-b394-34f1a23d01f3" (UID: "556e2861-dc40-424c-b394-34f1a23d01f3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:55:09.151887 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.151867    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-dshm" (OuterVolumeSpecName: "dshm") pod "556e2861-dc40-424c-b394-34f1a23d01f3" (UID: "556e2861-dc40-424c-b394-34f1a23d01f3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:55:09.151930 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.151881    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556e2861-dc40-424c-b394-34f1a23d01f3-kube-api-access-ql47n" (OuterVolumeSpecName: "kube-api-access-ql47n") pod "556e2861-dc40-424c-b394-34f1a23d01f3" (UID: "556e2861-dc40-424c-b394-34f1a23d01f3"). InnerVolumeSpecName "kube-api-access-ql47n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:55:09.250906 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.250866    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/556e2861-dc40-424c-b394-34f1a23d01f3-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:09.250906 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.250898    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ql47n\" (UniqueName: \"kubernetes.io/projected/556e2861-dc40-424c-b394-34f1a23d01f3-kube-api-access-ql47n\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:09.250906 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.250912    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/556e2861-dc40-424c-b394-34f1a23d01f3-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:55:09.433213 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.433174    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"]
Sep 19 12:55:09.436310 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:09.436286    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/sub-test-model-5d6124-kserve-6f7755c4b7-j9mxb"]
Sep 19 12:55:10.738795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:10.738768    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="556e2861-dc40-424c-b394-34f1a23d01f3" path="/var/lib/kubelet/pods/556e2861-dc40-424c-b394-34f1a23d01f3/volumes"
Sep 19 12:55:33.089254 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:33.089214    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:55:35.337823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.337792    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"]
Sep 19 12:55:35.338193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.338101    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77a94435-dabb-44e3-b2cf-8840c15e63c0" containerName="main"
Sep 19 12:55:35.338193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.338112    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="77a94435-dabb-44e3-b2cf-8840c15e63c0" containerName="main"
Sep 19 12:55:35.338193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.338121    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="556e2861-dc40-424c-b394-34f1a23d01f3" containerName="main"
Sep 19 12:55:35.338193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.338126    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="556e2861-dc40-424c-b394-34f1a23d01f3" containerName="main"
Sep 19 12:55:35.338193 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.338193    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="77a94435-dabb-44e3-b2cf-8840c15e63c0" containerName="main"
Sep 19 12:55:35.338362 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.338205    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="556e2861-dc40-424c-b394-34f1a23d01f3" containerName="main"
Sep 19 12:55:35.342377 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.342359    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.346267 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.346245    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"kube-root-ca.crt\""
Sep 19 12:55:35.346393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.346245    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"default-dockercfg-2cfx6\""
Sep 19 12:55:35.346393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.346245    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"openshift-service-ca.crt\""
Sep 19 12:55:35.346393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.346245    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-5d6124\"/\"rate-test-model-5d6124-kserve-self-signed-certs\""
Sep 19 12:55:35.350918 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.350896    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"]
Sep 19 12:55:35.357334 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.357312    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:55:35.471653 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.471628    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-home\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.471829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.471658    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjtlz\" (UniqueName: \"kubernetes.io/projected/ff007368-0dac-434b-a823-3c3e555321d9-kube-api-access-wjtlz\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.471829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.471680    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-model-cache\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.471829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.471769    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff007368-0dac-434b-a823-3c3e555321d9-tls-certs\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.471978 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.471836    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-dshm\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.471978 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.471883    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-tmp-dir\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572280 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572251    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-tmp-dir\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572291    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-home\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572308    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjtlz\" (UniqueName: \"kubernetes.io/projected/ff007368-0dac-434b-a823-3c3e555321d9-kube-api-access-wjtlz\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572328    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-model-cache\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572361    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff007368-0dac-434b-a823-3c3e555321d9-tls-certs\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572428 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572394    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-dshm\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572753 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572732    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-tmp-dir\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572810 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572740    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-home\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.572810 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.572776    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-model-cache\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.574708 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.574690    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-dshm\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.574986 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.574967    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff007368-0dac-434b-a823-3c3e555321d9-tls-certs\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.581791 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.581772    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjtlz\" (UniqueName: \"kubernetes.io/projected/ff007368-0dac-434b-a823-3c3e555321d9-kube-api-access-wjtlz\") pod \"rate-test-model-5d6124-kserve-69c6845475-7l8gp\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") " pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.651530 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.651476    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:35.919554 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:35.919519    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"]
Sep 19 12:55:35.921193 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:55:35.921163    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff007368_0dac_434b_a823_3c3e555321d9.slice/crio-536973aff7cbedd997b48459436d27fc9971c035ff31ea21d71ea0d3212b5fcf WatchSource:0}: Error finding container 536973aff7cbedd997b48459436d27fc9971c035ff31ea21d71ea0d3212b5fcf: Status 404 returned error can't find the container with id 536973aff7cbedd997b48459436d27fc9971c035ff31ea21d71ea0d3212b5fcf
Sep 19 12:55:36.244007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:36.243975    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp" event={"ID":"ff007368-0dac-434b-a823-3c3e555321d9","Type":"ContainerStarted","Data":"f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100"}
Sep 19 12:55:36.244157 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:36.244021    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp" event={"ID":"ff007368-0dac-434b-a823-3c3e555321d9","Type":"ContainerStarted","Data":"536973aff7cbedd997b48459436d27fc9971c035ff31ea21d71ea0d3212b5fcf"}
Sep 19 12:55:36.262328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:36.262282    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp" podStartSLOduration=1.262267703 podStartE2EDuration="1.262267703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:55:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:55:36.260302384 +0000 UTC m=+1819.620545326" watchObservedRunningTime="2026-09-19 12:55:36.262267703 +0000 UTC m=+1819.622510644"
Sep 19 12:55:39.141594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:39.141556    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:55:41.442950 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:41.442914    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:55:41.544219 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:41.544177    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:55:45.652197 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:45.652157    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:45.652197 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:45.652208    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:45.665130 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:45.665102    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:46.299258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:46.299225    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:55:51.025683 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:55:51.025643    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:56:06.228248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:06.228208    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:56:10.745439 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:10.745407    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:11.616335 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:11.616303    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"]
Sep 19 12:56:11.640846 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:11.640812    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:12.079347 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.079315    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:12.359796 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.359714    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"]
Sep 19 12:56:12.362899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.362875    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.365528 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.365497    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 19 12:56:12.365727 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.365712    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-f32129-openshift-default-dockercfg-lwhs2\""
Sep 19 12:56:12.374272 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.374249    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"]
Sep 19 12:56:12.386813 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.386784    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:12.487928 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.487888    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-credential-socket\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.487936    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrzmp\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-kube-api-access-lrzmp\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488001    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-istio-token\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488066    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-socket\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488122 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488110    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-envoy\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488134    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-data\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488159    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-certs\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488198    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6df700-301e-4da3-897e-0f8f196c8326-istio-podinfo\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.488330 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.488254    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6df700-301e-4da3-897e-0f8f196c8326-istiod-ca-cert\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589155 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589117    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrzmp\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-kube-api-access-lrzmp\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589173    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-istio-token\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589218    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-socket\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589264    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-envoy\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589288    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-data\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589314    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-certs\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589357 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589356    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6df700-301e-4da3-897e-0f8f196c8326-istio-podinfo\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.589692 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589388    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6df700-301e-4da3-897e-0f8f196c8326-istiod-ca-cert\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.590002 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.589969    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-credential-socket\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.590338 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.590174    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-data\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.591098    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-socket\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.591369    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6df700-301e-4da3-897e-0f8f196c8326-istiod-ca-cert\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.592753    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-certs\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.597719    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-envoy\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.597782    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-credential-socket\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.597930    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrzmp\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-kube-api-access-lrzmp\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.598700 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.598164    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6df700-301e-4da3-897e-0f8f196c8326-istio-podinfo\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.600178 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.600148    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-istio-token\") pod \"e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") " pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.674226 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.674141    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:12.719242 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.719194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:12.946444 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.946408    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"]
Sep 19 12:56:12.948153 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:56:12.948128    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa6df700_301e_4da3_897e_0f8f196c8326.slice/crio-bb26d21dca97c8da5debe689e6e324159301de8ab79767b07c254f0c4de2fefd WatchSource:0}: Error finding container bb26d21dca97c8da5debe689e6e324159301de8ab79767b07c254f0c4de2fefd: Status 404 returned error can't find the container with id bb26d21dca97c8da5debe689e6e324159301de8ab79767b07c254f0c4de2fefd
Sep 19 12:56:12.950291 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.950253    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:12.950373 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.950319    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:12.950373 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:12.950347    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:13.406516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:13.406480    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" event={"ID":"aa6df700-301e-4da3-897e-0f8f196c8326","Type":"ContainerStarted","Data":"be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"}
Sep 19 12:56:13.406911 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:13.406538    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" event={"ID":"aa6df700-301e-4da3-897e-0f8f196c8326","Type":"ContainerStarted","Data":"bb26d21dca97c8da5debe689e6e324159301de8ab79767b07c254f0c4de2fefd"}
Sep 19 12:56:13.427734 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:13.427666    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podStartSLOduration=1.427649234 podStartE2EDuration="1.427649234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:56:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:56:13.42605928 +0000 UTC m=+1856.786302221" watchObservedRunningTime="2026-09-19 12:56:13.427649234 +0000 UTC m=+1856.787892175"
Sep 19 12:56:13.674943 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:13.674865    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:14.432033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:14.431996    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:56:14.679638 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:14.679609    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:15.137414 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:15.137365    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:15.335233 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:15.335200    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:15.416506 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:15.416396    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:15.417304 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:15.417285    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:15.452005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:15.451953    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:16.367116 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:16.367073    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:17.186364 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.186319    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:17.363266 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.363226    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:17.773209 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.773161    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:17.873516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.873479    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:17.912359 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.912331    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"]
Sep 19 12:56:17.915726 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.915707    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:17.918559 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.918542    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f32129\""
Sep 19 12:56:17.918652 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.918546    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f32129-dockercfg-59jpq\""
Sep 19 12:56:17.925115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:17.925093    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"]
Sep 19 12:56:18.036520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.036430    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glxcb\" (UniqueName: \"kubernetes.io/projected/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-kube-api-access-glxcb\") pod \"payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.036520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.036511    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.137793 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.137750    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.137970 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.137868    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glxcb\" (UniqueName: \"kubernetes.io/projected/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-kube-api-access-glxcb\") pod \"payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.138480 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.138442    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.146722 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.146698    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glxcb\" (UniqueName: \"kubernetes.io/projected/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-kube-api-access-glxcb\") pod \"payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.227865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.227836    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:18.494375 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:18.494348    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"]
Sep 19 12:56:18.496729 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:56:18.496702    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4db129a_b9cc_4df9_93a8_b7cc7a0880ea.slice/crio-05b8bfdb56f1e6150b2392f5b6d1b9085ea5415cf44b8697d88ba9c9931ece73 WatchSource:0}: Error finding container 05b8bfdb56f1e6150b2392f5b6d1b9085ea5415cf44b8697d88ba9c9931ece73: Status 404 returned error can't find the container with id 05b8bfdb56f1e6150b2392f5b6d1b9085ea5415cf44b8697d88ba9c9931ece73
Sep 19 12:56:19.439471 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:19.439436    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd" event={"ID":"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea","Type":"ContainerStarted","Data":"4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c"}
Sep 19 12:56:19.439865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:19.439505    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd" event={"ID":"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea","Type":"ContainerStarted","Data":"05b8bfdb56f1e6150b2392f5b6d1b9085ea5415cf44b8697d88ba9c9931ece73"}
Sep 19 12:56:19.439865 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:19.439538    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:19.457271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:19.457221    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd" podStartSLOduration=2.457207292 podStartE2EDuration="2.457207292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:56:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:56:19.456022815 +0000 UTC m=+1862.816265757" watchObservedRunningTime="2026-09-19 12:56:19.457207292 +0000 UTC m=+1862.817450257"
Sep 19 12:56:26.635386 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:26.635332    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp" podUID="ff007368-0dac-434b-a823-3c3e555321d9" containerName="main" containerID="cri-o://f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100" gracePeriod=45
Sep 19 12:56:26.910093 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:26.910070    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:56:27.022215 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022187    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-home\") pod \"ff007368-0dac-434b-a823-3c3e555321d9\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") "
Sep 19 12:56:27.022418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022228    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff007368-0dac-434b-a823-3c3e555321d9-tls-certs\") pod \"ff007368-0dac-434b-a823-3c3e555321d9\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") "
Sep 19 12:56:27.022418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022249    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-dshm\") pod \"ff007368-0dac-434b-a823-3c3e555321d9\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") "
Sep 19 12:56:27.022418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022275    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjtlz\" (UniqueName: \"kubernetes.io/projected/ff007368-0dac-434b-a823-3c3e555321d9-kube-api-access-wjtlz\") pod \"ff007368-0dac-434b-a823-3c3e555321d9\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") "
Sep 19 12:56:27.022418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022323    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-model-cache\") pod \"ff007368-0dac-434b-a823-3c3e555321d9\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") "
Sep 19 12:56:27.022418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022351    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-tmp-dir\") pod \"ff007368-0dac-434b-a823-3c3e555321d9\" (UID: \"ff007368-0dac-434b-a823-3c3e555321d9\") "
Sep 19 12:56:27.022714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022475    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-home" (OuterVolumeSpecName: "home") pod "ff007368-0dac-434b-a823-3c3e555321d9" (UID: "ff007368-0dac-434b-a823-3c3e555321d9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:56:27.022714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022596    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff007368-0dac-434b-a823-3c3e555321d9" (UID: "ff007368-0dac-434b-a823-3c3e555321d9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:56:27.022714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022679    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:56:27.022714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022695    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:56:27.022714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.022692    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff007368-0dac-434b-a823-3c3e555321d9" (UID: "ff007368-0dac-434b-a823-3c3e555321d9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:56:27.024429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.024398    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff007368-0dac-434b-a823-3c3e555321d9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff007368-0dac-434b-a823-3c3e555321d9" (UID: "ff007368-0dac-434b-a823-3c3e555321d9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:56:27.024429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.024414    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-dshm" (OuterVolumeSpecName: "dshm") pod "ff007368-0dac-434b-a823-3c3e555321d9" (UID: "ff007368-0dac-434b-a823-3c3e555321d9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:56:27.024429 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.024421    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff007368-0dac-434b-a823-3c3e555321d9-kube-api-access-wjtlz" (OuterVolumeSpecName: "kube-api-access-wjtlz") pod "ff007368-0dac-434b-a823-3c3e555321d9" (UID: "ff007368-0dac-434b-a823-3c3e555321d9"). InnerVolumeSpecName "kube-api-access-wjtlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:56:27.123567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.123537    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjtlz\" (UniqueName: \"kubernetes.io/projected/ff007368-0dac-434b-a823-3c3e555321d9-kube-api-access-wjtlz\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:56:27.123567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.123562    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:56:27.123567 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.123571    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff007368-0dac-434b-a823-3c3e555321d9-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:56:27.123795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.123581    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff007368-0dac-434b-a823-3c3e555321d9-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:56:27.483145 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.483108    2571 generic.go:356] "Generic (PLEG): container finished" podID="ff007368-0dac-434b-a823-3c3e555321d9" containerID="f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100" exitCode=0
Sep 19 12:56:27.483387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.483151    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"
Sep 19 12:56:27.483387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.483195    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp" event={"ID":"ff007368-0dac-434b-a823-3c3e555321d9","Type":"ContainerDied","Data":"f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100"}
Sep 19 12:56:27.483387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.483233    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp" event={"ID":"ff007368-0dac-434b-a823-3c3e555321d9","Type":"ContainerDied","Data":"536973aff7cbedd997b48459436d27fc9971c035ff31ea21d71ea0d3212b5fcf"}
Sep 19 12:56:27.483387 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.483252    2571 scope.go:117] "RemoveContainer" containerID="f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100"
Sep 19 12:56:27.492932 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.492917    2571 scope.go:117] "RemoveContainer" containerID="f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100"
Sep 19 12:56:27.493194 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:56:27.493176    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100\": container with ID starting with f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100 not found: ID does not exist" containerID="f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100"
Sep 19 12:56:27.493251 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.493201    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100"} err="failed to get container status \"f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100\": rpc error: code = NotFound desc = could not find container \"f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100\": container with ID starting with f448a83c7984b95d4eb9c80e031f14e0bb51c46027c3379b2196ea9a7f89c100 not found: ID does not exist"
Sep 19 12:56:27.505258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.505237    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"]
Sep 19 12:56:27.509455 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.509430    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-5d6124/rate-test-model-5d6124-kserve-69c6845475-7l8gp"]
Sep 19 12:56:27.627588 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:27.627553    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:56:28.748328 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:28.748298    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff007368-0dac-434b-a823-3c3e555321d9" path="/var/lib/kubelet/pods/ff007368-0dac-434b-a823-3c3e555321d9/volumes"
Sep 19 12:56:30.446249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:30.446149    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:56:31.458630 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:31.458583    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 12:56:31.459143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:31.458650    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 12:56:32.513645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:32.513611    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa6df700-301e-4da3-897e-0f8f196c8326" containerID="be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4" exitCode=0
Sep 19 12:56:32.513645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:32.513646    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" event={"ID":"aa6df700-301e-4da3-897e-0f8f196c8326","Type":"ContainerDied","Data":"be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"}
Sep 19 12:56:32.514101 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:32.513992    2571 scope.go:117] "RemoveContainer" containerID="be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"
Sep 19 12:56:32.515117 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:32.515085    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:32.515185 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:32.515172    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:32.515227 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:32.515205    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:33.520349 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:33.520314    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" event={"ID":"aa6df700-301e-4da3-897e-0f8f196c8326","Type":"ContainerStarted","Data":"81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7"}
Sep 19 12:56:33.674902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:33.674867    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:33.675114 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:33.675092    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 19 12:56:33.675160 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:33.675139    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 19 12:56:34.674633 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:34.674590    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 19 12:56:34.675022 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:34.674652    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 19 12:56:35.675400 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:35.675362    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 19 12:56:35.675826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:35.675420    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 19 12:56:36.674582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:36.674544    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 19 12:56:36.674799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:36.674616    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 19 12:56:37.675307 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:37.675271    2571 patch_prober.go:28] interesting pod/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused" start-of-body=
Sep 19 12:56:37.675823 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:37.675339    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.71:15021/healthz/ready\": dial tcp 10.134.0.71:15021: connect: connection refused"
Sep 19 12:56:38.678035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:38.678006    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:38.678419 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:38.678363    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:38.678990 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:38.678974    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 12:56:40.940164 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:40.940129    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:41.765121 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:41.765072    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:42.538313 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:42.538275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:43.200686 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:43.200652    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:45.401475 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:45.401432    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:46.101764 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.101732    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:46.537455 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.537421    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"]
Sep 19 12:56:46.537948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.537753    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff007368-0dac-434b-a823-3c3e555321d9" containerName="main"
Sep 19 12:56:46.537948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.537764    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff007368-0dac-434b-a823-3c3e555321d9" containerName="main"
Sep 19 12:56:46.537948 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.537830    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff007368-0dac-434b-a823-3c3e555321d9" containerName="main"
Sep 19 12:56:46.539693 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.539674    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.542976 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.542956    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-4e754a-openshift-default-dockercfg-hdj4n\""
Sep 19 12:56:46.557048 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.557028    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"]
Sep 19 12:56:46.583258 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.583217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:46.699701 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.699663    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72dd51a5-5996-484c-8843-39d4606d2ba3-istiod-ca-cert\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.699971    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-certs\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700127 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700113    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-envoy\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700262 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700247    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-credential-socket\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700402    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-data\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700576 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700446    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dd99\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-kube-api-access-5dd99\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700576 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700539    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-socket\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700679 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700574    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-token\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.700679 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.700602    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-podinfo\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801480 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801371    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-credential-socket\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801497    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-data\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801579    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dd99\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-kube-api-access-5dd99\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801632    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-socket\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801656    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-token\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801701    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-podinfo\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801754    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-data\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801757    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72dd51a5-5996-484c-8843-39d4606d2ba3-istiod-ca-cert\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801797    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-credential-socket\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801823    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-certs\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.801857 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.801840    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-envoy\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.802244 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.802214    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-socket\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.802617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.802590    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-certs\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.802740 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.802592    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72dd51a5-5996-484c-8843-39d4606d2ba3-istiod-ca-cert\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.804222 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.804197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:46.804511 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.804491    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-envoy\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.804901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.804885    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-podinfo\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.819899 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.819872    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-token\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.820161 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.820146    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dd99\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-kube-api-access-5dd99\") pod \"e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") " pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:46.851103 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:46.851075    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:47.126322 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.126297    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"]
Sep 19 12:56:47.128269 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:56:47.128242    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72dd51a5_5996_484c_8843_39d4606d2ba3.slice/crio-25ee538e065645650325fa8ec9d41178edfad89afbf57c21d3064315c5fe8dcb WatchSource:0}: Error finding container 25ee538e065645650325fa8ec9d41178edfad89afbf57c21d3064315c5fe8dcb: Status 404 returned error can't find the container with id 25ee538e065645650325fa8ec9d41178edfad89afbf57c21d3064315c5fe8dcb
Sep 19 12:56:47.130437 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.130399    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:47.130543 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.130510    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:47.130609 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.130555    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:56:47.597830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.597794    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" event={"ID":"72dd51a5-5996-484c-8843-39d4606d2ba3","Type":"ContainerStarted","Data":"3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"}
Sep 19 12:56:47.598249 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.597849    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" event={"ID":"72dd51a5-5996-484c-8843-39d4606d2ba3","Type":"ContainerStarted","Data":"25ee538e065645650325fa8ec9d41178edfad89afbf57c21d3064315c5fe8dcb"}
Sep 19 12:56:47.851812 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:47.851709    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:48.856946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:48.856916    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:48.885694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:48.885640    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podStartSLOduration=2.885623591 podStartE2EDuration="2.885623591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:56:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:56:47.622826783 +0000 UTC m=+1890.983069724" watchObservedRunningTime="2026-09-19 12:56:48.885623591 +0000 UTC m=+1892.245866531"
Sep 19 12:56:49.413089 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:49.413054    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:49.584569 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:49.584527    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:49.606946 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:49.606921    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:49.607792 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:49.607773    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:56:49.687599 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:49.687506    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:50.273319 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:50.273281    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:51.009868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:51.009830    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:51.785795 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:51.785756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:51.887255 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:51.887216    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:51.987252 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:51.987217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:52.248143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.248104    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:52.392417 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.392390    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"]
Sep 19 12:56:52.392650 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.392623    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:56:52.395184 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.395164    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.398248 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.398228    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-4e754a-dockercfg-vgdc7\""
Sep 19 12:56:52.398360 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.398252    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-4e754a\""
Sep 19 12:56:52.410627 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.410608    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"]
Sep 19 12:56:52.446838 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.446815    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s24p\" (UniqueName: \"kubernetes.io/projected/cf0e09bc-5877-4cdc-86db-517086a9485d-kube-api-access-4s24p\") pod \"payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.446947 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.446886    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf0e09bc-5877-4cdc-86db-517086a9485d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.547776 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.547688    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s24p\" (UniqueName: \"kubernetes.io/projected/cf0e09bc-5877-4cdc-86db-517086a9485d-kube-api-access-4s24p\") pod \"payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.547931 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.547783    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf0e09bc-5877-4cdc-86db-517086a9485d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.548485 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.548445    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf0e09bc-5877-4cdc-86db-517086a9485d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.556005 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.555985    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s24p\" (UniqueName: \"kubernetes.io/projected/cf0e09bc-5877-4cdc-86db-517086a9485d-kube-api-access-4s24p\") pod \"payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.706035 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.706008    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:52.974384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:52.974361    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"]
Sep 19 12:56:52.976000 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:56:52.975976    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf0e09bc_5877_4cdc_86db_517086a9485d.slice/crio-a947aa6c1e094641b438f1ffa8e215ef5e28b82a9888375da9964e207b52bfd6 WatchSource:0}: Error finding container a947aa6c1e094641b438f1ffa8e215ef5e28b82a9888375da9964e207b52bfd6: Status 404 returned error can't find the container with id a947aa6c1e094641b438f1ffa8e215ef5e28b82a9888375da9964e207b52bfd6
Sep 19 12:56:53.626614 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:53.626575    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8" event={"ID":"cf0e09bc-5877-4cdc-86db-517086a9485d","Type":"ContainerStarted","Data":"33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31"}
Sep 19 12:56:53.626801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:53.626642    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:56:53.626801 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:53.626669    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8" event={"ID":"cf0e09bc-5877-4cdc-86db-517086a9485d","Type":"ContainerStarted","Data":"a947aa6c1e094641b438f1ffa8e215ef5e28b82a9888375da9964e207b52bfd6"}
Sep 19 12:56:53.644822 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:53.644620    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8" podStartSLOduration=1.644602004 podStartE2EDuration="1.644602004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:56:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:56:53.644584363 +0000 UTC m=+1897.004827302" watchObservedRunningTime="2026-09-19 12:56:53.644602004 +0000 UTC m=+1897.004844945"
Sep 19 12:56:57.534972 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:56:57.534936    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:57:04.633921 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:04.633883    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 12:57:05.648643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:05.648605    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 19 12:57:05.649062 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:05.648695    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 19 12:57:07.702626 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:07.702592    2571 generic.go:356] "Generic (PLEG): container finished" podID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerID="3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b" exitCode=0
Sep 19 12:57:07.703045 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:07.702665    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" event={"ID":"72dd51a5-5996-484c-8843-39d4606d2ba3","Type":"ContainerDied","Data":"3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"}
Sep 19 12:57:07.703113 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:07.703096    2571 scope.go:117] "RemoveContainer" containerID="3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"
Sep 19 12:57:07.704108 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:07.704081    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:57:07.704165 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:07.704147    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:57:07.704201 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:07.704175    2571 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 19 12:57:08.709509 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:08.709475    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" event={"ID":"72dd51a5-5996-484c-8843-39d4606d2ba3","Type":"ContainerStarted","Data":"f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737"}
Sep 19 12:57:08.852309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:08.852269    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:57:08.852605 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:08.852577    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 19 12:57:08.852814 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:08.852788    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 19 12:57:09.851914 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:09.851871    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 19 12:57:09.852435 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:09.851936    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 19 12:57:10.851828 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:10.851786    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 19 12:57:10.852115 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:10.852085    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 19 12:57:11.852055 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:11.851974    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 19 12:57:11.852055 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:11.852041    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 19 12:57:12.852009 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:12.851962    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 19 12:57:12.852224 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:12.852030    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 19 12:57:13.852384 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:13.852347    2571 patch_prober.go:28] interesting pod/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Sep 19 12:57:13.852830 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:13.852403    2571 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Sep 19 12:57:14.909634 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:14.909603    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:57:14.910136 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:14.909986    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:57:14.910799 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:14.910779    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 12:57:15.475437 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:15.475404    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:16.287232 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:16.287194    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:16.956078 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:16.956050    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:17.775202 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:17.775166    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:18.440560 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:18.440512    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:19.682590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:19.682554    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:24.963230 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:24.963198    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8_cf0e09bc-5877-4cdc-86db-517086a9485d/payload-processing/0.log"
Sep 19 12:57:32.724534 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.724504    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"]
Sep 19 12:57:32.729356 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.729326    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:32.733714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.733681    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f32129\"/\"default-dockercfg-dtt6z\""
Sep 19 12:57:32.733714 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.733686    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-f32129\"/\"ipp-route-f32129-kserve-self-signed-certs\""
Sep 19 12:57:32.733902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.733686    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f32129\"/\"openshift-service-ca.crt\""
Sep 19 12:57:32.733902 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.733838    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-f32129\"/\"kube-root-ca.crt\""
Sep 19 12:57:32.740437 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.740417    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"]
Sep 19 12:57:32.753226 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.753203    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:32.917170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.917130    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-dshm\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:32.917170 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.917168    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-home\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:32.917418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.917232    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdw9l\" (UniqueName: \"kubernetes.io/projected/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-kube-api-access-xdw9l\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:32.917418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.917284    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-model-cache\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:32.917418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.917308    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tmp-dir\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:32.917418 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:32.917335    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tls-certs\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.017997    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdw9l\" (UniqueName: \"kubernetes.io/projected/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-kube-api-access-xdw9l\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018051    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-model-cache\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018082    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tmp-dir\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018123    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tls-certs\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018167    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-dshm\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018292 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018197    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-home\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018562 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018530    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tmp-dir\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018630 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018577    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-model-cache\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.018630 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.018622    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-home\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.020397 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.020377    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-dshm\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.020725 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.020707    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tls-certs\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.026044 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.026020    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdw9l\" (UniqueName: \"kubernetes.io/projected/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-kube-api-access-xdw9l\") pod \"ipp-route-f32129-kserve-85bf75dcb7-vd7bb\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") " pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.040040 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.040023    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:33.309482 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.309385    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"]
Sep 19 12:57:33.312548 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:57:33.312520    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3d9d0109_415e_4d7b_8bd7_d797c1dc4aff.slice/crio-e4b83cf99a4f29f55a359959571e1512f9d056c5a1296fa90452ebbb055dbcbb WatchSource:0}: Error finding container e4b83cf99a4f29f55a359959571e1512f9d056c5a1296fa90452ebbb055dbcbb: Status 404 returned error can't find the container with id e4b83cf99a4f29f55a359959571e1512f9d056c5a1296fa90452ebbb055dbcbb
Sep 19 12:57:33.827982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.827945    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb" event={"ID":"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff","Type":"ContainerStarted","Data":"8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1"}
Sep 19 12:57:33.827982 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.827986    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb" event={"ID":"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff","Type":"ContainerStarted","Data":"e4b83cf99a4f29f55a359959571e1512f9d056c5a1296fa90452ebbb055dbcbb"}
Sep 19 12:57:33.846853 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:33.846804    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb" podStartSLOduration=1.846788313 podStartE2EDuration="1.846788313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:57:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:57:33.845234456 +0000 UTC m=+1937.205477398" watchObservedRunningTime="2026-09-19 12:57:33.846788313 +0000 UTC m=+1937.207031253"
Sep 19 12:57:37.137694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:37.137661    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:57:43.040708 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:43.040670    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:43.040708 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:43.040719    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:43.053082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:43.053051    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:43.887057 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:43.887030    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:57:51.169425 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:51.169388    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:57:54.430868 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:54.430836    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:57:58.850967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:57:58.850909    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:00.942303 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:00.942221    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd_f4db129a-b9cc-4df9-93a8-b7cc7a0880ea/payload-processing/0.log"
Sep 19 12:58:01.161922 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:01.161895    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8_cf0e09bc-5877-4cdc-86db-517086a9485d/payload-processing/0.log"
Sep 19 12:58:01.282223 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:01.282186    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8_cf0e09bc-5877-4cdc-86db-517086a9485d/payload-processing/0.log"
Sep 19 12:58:01.547687 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:01.547596    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:01.845749 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:01.845662    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:02.050490 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:02.050441    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:04.494641 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:04.494607    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:04.706245 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:04.706202    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:04.806348 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:04.806259    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:05.465814 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:05.465783    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:06.159877 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:06.159840    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:06.871578 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:06.871542    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:07.879371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:07.879341    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:07.939742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:07.939700    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:08.040643 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.040592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:08.451341 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.451307    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:08.550235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.550197    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:08.624154 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.624125    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"]
Sep 19 12:58:08.627645 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.627628    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.630125 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.630103    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-1593f0-dockercfg-bl7nb\""
Sep 19 12:58:08.630243 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.630179    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-1593f0\""
Sep 19 12:58:08.637064 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.637040    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"]
Sep 19 12:58:08.732694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.732607    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dmxl\" (UniqueName: \"kubernetes.io/projected/80122ade-aaba-42fe-a435-c6316666fa79-kube-api-access-7dmxl\") pod \"payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.732694 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.732686    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80122ade-aaba-42fe-a435-c6316666fa79-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.833667 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.833629    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dmxl\" (UniqueName: \"kubernetes.io/projected/80122ade-aaba-42fe-a435-c6316666fa79-kube-api-access-7dmxl\") pod \"payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.833881 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.833799    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80122ade-aaba-42fe-a435-c6316666fa79-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.834518 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.834499    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80122ade-aaba-42fe-a435-c6316666fa79-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.842200 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.842176    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dmxl\" (UniqueName: \"kubernetes.io/projected/80122ade-aaba-42fe-a435-c6316666fa79-kube-api-access-7dmxl\") pod \"payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:08.939491 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:08.939437    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:09.209220 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:09.209136    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"]
Sep 19 12:58:09.212085 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:58:09.212060    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80122ade_aaba_42fe_a435_c6316666fa79.slice/crio-b5e163489f07050f502ee280c9373a65d8c733faf24bf7cb0f2060898ac4ffa4 WatchSource:0}: Error finding container b5e163489f07050f502ee280c9373a65d8c733faf24bf7cb0f2060898ac4ffa4: Status 404 returned error can't find the container with id b5e163489f07050f502ee280c9373a65d8c733faf24bf7cb0f2060898ac4ffa4
Sep 19 12:58:09.999806 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:09.999770    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk" event={"ID":"80122ade-aaba-42fe-a435-c6316666fa79","Type":"ContainerStarted","Data":"07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b"}
Sep 19 12:58:10.000209 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:09.999824    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk" event={"ID":"80122ade-aaba-42fe-a435-c6316666fa79","Type":"ContainerStarted","Data":"b5e163489f07050f502ee280c9373a65d8c733faf24bf7cb0f2060898ac4ffa4"}
Sep 19 12:58:10.000209 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:09.999889    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:10.016215 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:10.016172    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk" podStartSLOduration=2.016158633 podStartE2EDuration="2.016158633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 12:58:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 12:58:10.014764424 +0000 UTC m=+1973.375007366" watchObservedRunningTime="2026-09-19 12:58:10.016158633 +0000 UTC m=+1973.376401573"
Sep 19 12:58:13.929014 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:13.928979    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:58:21.007095 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:21.007064    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:30.463256 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:30.463220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:31.159316 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:31.159277    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:31.830659 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:31.830625    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:32.110485 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:32.110380    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:32.814501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:32.814449    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:33.448761 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:33.448729    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:34.252049 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:34.252014    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:34.842673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:34.842634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:36.275661 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:36.275621    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:37.067918 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:37.067891    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:37.858721 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:37.858668    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:38.658056 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:38.658021    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:39.454201 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:39.454159    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:40.265850 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:40.265811    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:46.475967 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.475932    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:46.518171 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.518127    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"]
Sep 19 12:58:46.518438 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.518412    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk" podUID="80122ade-aaba-42fe-a435-c6316666fa79" containerName="payload-pre-processing" containerID="cri-o://07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b" gracePeriod=30
Sep 19 12:58:46.555336 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.555292    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:46.664673 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.664634    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:46.730520 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.730405    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:46.769054 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.769031    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:46.881259 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.881224    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80122ade-aaba-42fe-a435-c6316666fa79-plugins-config-volume\") pod \"80122ade-aaba-42fe-a435-c6316666fa79\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") "
Sep 19 12:58:46.881501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.881276    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dmxl\" (UniqueName: \"kubernetes.io/projected/80122ade-aaba-42fe-a435-c6316666fa79-kube-api-access-7dmxl\") pod \"80122ade-aaba-42fe-a435-c6316666fa79\" (UID: \"80122ade-aaba-42fe-a435-c6316666fa79\") "
Sep 19 12:58:46.881680 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.881654    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80122ade-aaba-42fe-a435-c6316666fa79-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80122ade-aaba-42fe-a435-c6316666fa79" (UID: "80122ade-aaba-42fe-a435-c6316666fa79"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:58:46.883394 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.883372    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80122ade-aaba-42fe-a435-c6316666fa79-kube-api-access-7dmxl" (OuterVolumeSpecName: "kube-api-access-7dmxl") pod "80122ade-aaba-42fe-a435-c6316666fa79" (UID: "80122ade-aaba-42fe-a435-c6316666fa79"). InnerVolumeSpecName "kube-api-access-7dmxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:58:46.982629 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.982547    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80122ade-aaba-42fe-a435-c6316666fa79-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:58:46.982629 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:46.982576    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dmxl\" (UniqueName: \"kubernetes.io/projected/80122ade-aaba-42fe-a435-c6316666fa79-kube-api-access-7dmxl\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:58:47.093317 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.093273    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:47.179235 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.179196    2571 generic.go:356] "Generic (PLEG): container finished" podID="80122ade-aaba-42fe-a435-c6316666fa79" containerID="07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b" exitCode=0
Sep 19 12:58:47.179423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.179240    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"
Sep 19 12:58:47.179423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.179301    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk" event={"ID":"80122ade-aaba-42fe-a435-c6316666fa79","Type":"ContainerDied","Data":"07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b"}
Sep 19 12:58:47.179423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.179363    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk" event={"ID":"80122ade-aaba-42fe-a435-c6316666fa79","Type":"ContainerDied","Data":"b5e163489f07050f502ee280c9373a65d8c733faf24bf7cb0f2060898ac4ffa4"}
Sep 19 12:58:47.179423 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.179398    2571 scope.go:117] "RemoveContainer" containerID="07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b"
Sep 19 12:58:47.180426 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.180363    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:47.188143 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.188125    2571 scope.go:117] "RemoveContainer" containerID="07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b"
Sep 19 12:58:47.188402 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:58:47.188385    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b\": container with ID starting with 07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b not found: ID does not exist" containerID="07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b"
Sep 19 12:58:47.188445 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.188408    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b"} err="failed to get container status \"07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b\": rpc error: code = NotFound desc = could not find container \"07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b\": container with ID starting with 07f87dc7a8bdafa3f1948a70a2a062509ff619e52b6a3c5204f68a02f78b5d1b not found: ID does not exist"
Sep 19 12:58:47.203501 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.203480    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"]
Sep 19 12:58:47.206867 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.206845    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-1593f0-5db7f9f558-52kvk"]
Sep 19 12:58:47.945620 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:47.945586    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:48.740516 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:48.740479    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80122ade-aaba-42fe-a435-c6316666fa79" path="/var/lib/kubelet/pods/80122ade-aaba-42fe-a435-c6316666fa79/volumes"
Sep 19 12:58:48.868803 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:48.868771    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:49.557831 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:49.557801    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:50.247371 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:50.247324    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:51.126282 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:51.126251    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:58:53.138678 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:53.138644    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:53.248649 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:53.248608    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:53.508495 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:53.508439    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:56.911481 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.911427    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"]
Sep 19 12:58:56.911851 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.911780    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80122ade-aaba-42fe-a435-c6316666fa79" containerName="payload-pre-processing"
Sep 19 12:58:56.911851 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.911793    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="80122ade-aaba-42fe-a435-c6316666fa79" containerName="payload-pre-processing"
Sep 19 12:58:56.911851 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.911846    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="80122ade-aaba-42fe-a435-c6316666fa79" containerName="payload-pre-processing"
Sep 19 12:58:56.916231 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.916208    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:58:56.918699 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.918680    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 12:58:56.918829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.918760    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 12:58:56.923043 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:56.923020    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"]
Sep 19 12:58:57.065684 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:57.065653    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7d42d-d616-4fb9-8f25-a2bf8af580ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr\" (UID: \"1db7d42d-d616-4fb9-8f25-a2bf8af580ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:58:57.166321 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:57.166241    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7d42d-d616-4fb9-8f25-a2bf8af580ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr\" (UID: \"1db7d42d-d616-4fb9-8f25-a2bf8af580ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:58:57.166855 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:57.166836    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7d42d-d616-4fb9-8f25-a2bf8af580ca-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr\" (UID: \"1db7d42d-d616-4fb9-8f25-a2bf8af580ca\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:58:57.227901 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:57.227876    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:58:57.319028 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:57.318993    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:57.498128 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:57.498107    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"]
Sep 19 12:58:57.500283 ip-10-0-131-60 kubenswrapper[2571]: W0919 12:58:57.500255    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1db7d42d_d616_4fb9_8f25_a2bf8af580ca.slice/crio-0e11463a03a9f308d67fe7a25e720da0ef1cc8b162ddeaf07679ccfa58b2d37f WatchSource:0}: Error finding container 0e11463a03a9f308d67fe7a25e720da0ef1cc8b162ddeaf07679ccfa58b2d37f: Status 404 returned error can't find the container with id 0e11463a03a9f308d67fe7a25e720da0ef1cc8b162ddeaf07679ccfa58b2d37f
Sep 19 12:58:58.095218 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:58.095133    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:58.234023 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:58.233995    2571 generic.go:356] "Generic (PLEG): container finished" podID="1db7d42d-d616-4fb9-8f25-a2bf8af580ca" containerID="b85d373d35be0d832a425b60ddeabc1ed2c4b104ce825a37721c84eeb193cca1" exitCode=0
Sep 19 12:58:58.234146 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:58.234078    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr" event={"ID":"1db7d42d-d616-4fb9-8f25-a2bf8af580ca","Type":"ContainerDied","Data":"b85d373d35be0d832a425b60ddeabc1ed2c4b104ce825a37721c84eeb193cca1"}
Sep 19 12:58:58.234146 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:58.234128    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr" event={"ID":"1db7d42d-d616-4fb9-8f25-a2bf8af580ca","Type":"ContainerStarted","Data":"0e11463a03a9f308d67fe7a25e720da0ef1cc8b162ddeaf07679ccfa58b2d37f"}
Sep 19 12:58:58.975670 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:58.975633    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:58:59.365306 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:59.365283    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:58:59.487545 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:59.487524    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7d42d-d616-4fb9-8f25-a2bf8af580ca-maas-api-service-ca\") pod \"1db7d42d-d616-4fb9-8f25-a2bf8af580ca\" (UID: \"1db7d42d-d616-4fb9-8f25-a2bf8af580ca\") "
Sep 19 12:58:59.487847 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:59.487825    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1db7d42d-d616-4fb9-8f25-a2bf8af580ca-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1db7d42d-d616-4fb9-8f25-a2bf8af580ca" (UID: "1db7d42d-d616-4fb9-8f25-a2bf8af580ca"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:58:59.588585 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:59.588529    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1db7d42d-d616-4fb9-8f25-a2bf8af580ca-maas-api-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:58:59.672053 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:58:59.672020    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:00.243449 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:00.243421    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"
Sep 19 12:59:00.243617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:00.243452    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr" event={"ID":"1db7d42d-d616-4fb9-8f25-a2bf8af580ca","Type":"ContainerDied","Data":"0e11463a03a9f308d67fe7a25e720da0ef1cc8b162ddeaf07679ccfa58b2d37f"}
Sep 19 12:59:00.243617 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:00.243509    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0e11463a03a9f308d67fe7a25e720da0ef1cc8b162ddeaf07679ccfa58b2d37f"
Sep 19 12:59:00.364003 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:00.363973    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:01.205712 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:01.205677    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:09.939347 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:09.939309    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:59:12.535120 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.535082    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:12.557229 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.557198    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"]
Sep 19 12:59:12.557498 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.557452    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd" podUID="f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" containerName="payload-processing" containerID="cri-o://4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c" gracePeriod=30
Sep 19 12:59:12.633797 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.633762    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:12.736393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.736360    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:12.742590 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.742562    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"]
Sep 19 12:59:12.742736 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.742601    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-f32129-w2czr"]
Sep 19 12:59:12.809309 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.809264    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:12.814025 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.814002    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:59:12.908393 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.908360    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-plugins-config-volume\") pod \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") "
Sep 19 12:59:12.908594 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.908485    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glxcb\" (UniqueName: \"kubernetes.io/projected/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-kube-api-access-glxcb\") pod \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\" (UID: \"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea\") "
Sep 19 12:59:12.908820 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.908796    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" (UID: "f4db129a-b9cc-4df9-93a8-b7cc7a0880ea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 12:59:12.910538 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:12.910516    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-kube-api-access-glxcb" (OuterVolumeSpecName: "kube-api-access-glxcb") pod "f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" (UID: "f4db129a-b9cc-4df9-93a8-b7cc7a0880ea"). InnerVolumeSpecName "kube-api-access-glxcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:59:13.009844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.009801    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:13.009844 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.009836    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glxcb\" (UniqueName: \"kubernetes.io/projected/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea-kube-api-access-glxcb\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:13.114352 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.114275    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:13.215206 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.215168    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:13.304829 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.304796    2571 generic.go:356] "Generic (PLEG): container finished" podID="f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" containerID="4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c" exitCode=0
Sep 19 12:59:13.305007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.304842    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"
Sep 19 12:59:13.305007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.304885    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd" event={"ID":"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea","Type":"ContainerDied","Data":"4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c"}
Sep 19 12:59:13.305007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.304935    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd" event={"ID":"f4db129a-b9cc-4df9-93a8-b7cc7a0880ea","Type":"ContainerDied","Data":"05b8bfdb56f1e6150b2392f5b6d1b9085ea5415cf44b8697d88ba9c9931ece73"}
Sep 19 12:59:13.305007 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.304959    2571 scope.go:117] "RemoveContainer" containerID="4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c"
Sep 19 12:59:13.313682 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.313667    2571 scope.go:117] "RemoveContainer" containerID="4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c"
Sep 19 12:59:13.313936 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:59:13.313920    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c\": container with ID starting with 4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c not found: ID does not exist" containerID="4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c"
Sep 19 12:59:13.313980 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.313942    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c"} err="failed to get container status \"4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c\": rpc error: code = NotFound desc = could not find container \"4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c\": container with ID starting with 4f02dc7e5b141c441732b41b8fb8ed34b803c185be0c5046521bac0e9bc71b6c not found: ID does not exist"
Sep 19 12:59:13.329647 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.329626    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"]
Sep 19 12:59:13.332724 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.332703    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-f32129-8668476895-kg9xd"]
Sep 19 12:59:13.895074 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:13.895035    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:14.739180 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:14.739149    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1db7d42d-d616-4fb9-8f25-a2bf8af580ca" path="/var/lib/kubelet/pods/1db7d42d-d616-4fb9-8f25-a2bf8af580ca/volumes"
Sep 19 12:59:14.739561 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:14.739547    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" path="/var/lib/kubelet/pods/f4db129a-b9cc-4df9-93a8-b7cc7a0880ea/volumes"
Sep 19 12:59:14.870789 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:14.870756    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:15.580271 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:15.580238    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:18.096361 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:18.096329    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"]
Sep 19 12:59:18.150623 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:18.150592    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:23.334759 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:23.334723    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 12:59:26.649064 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:26.649028    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:26.750582 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:26.750543    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:33.115742 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.115650    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb" podUID="3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" containerName="main" containerID="cri-o://8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1" gracePeriod=45
Sep 19 12:59:33.367561 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.367501    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:59:33.399033 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.399002    2571 generic.go:356] "Generic (PLEG): container finished" podID="3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" containerID="8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1" exitCode=0
Sep 19 12:59:33.399175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.399041    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb" event={"ID":"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff","Type":"ContainerDied","Data":"8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1"}
Sep 19 12:59:33.399175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.399070    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb" event={"ID":"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff","Type":"ContainerDied","Data":"e4b83cf99a4f29f55a359959571e1512f9d056c5a1296fa90452ebbb055dbcbb"}
Sep 19 12:59:33.399175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.399074    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"
Sep 19 12:59:33.399175 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.399092    2571 scope.go:117] "RemoveContainer" containerID="8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1"
Sep 19 12:59:33.408745 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.408725    2571 scope.go:117] "RemoveContainer" containerID="8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1"
Sep 19 12:59:33.409030 ip-10-0-131-60 kubenswrapper[2571]: E0919 12:59:33.409013    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1\": container with ID starting with 8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1 not found: ID does not exist" containerID="8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1"
Sep 19 12:59:33.409092 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.409041    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1"} err="failed to get container status \"8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1\": rpc error: code = NotFound desc = could not find container \"8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1\": container with ID starting with 8d2925185783a151e8fb3e21f05ba7cbd2a3639e6d8b31b0f895c25c5a25fba1 not found: ID does not exist"
Sep 19 12:59:33.486288 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486251    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-home\") pod \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") "
Sep 19 12:59:33.486453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486344    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tls-certs\") pod \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") "
Sep 19 12:59:33.486453 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486407    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-model-cache\") pod \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") "
Sep 19 12:59:33.486552 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486448    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdw9l\" (UniqueName: \"kubernetes.io/projected/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-kube-api-access-xdw9l\") pod \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") "
Sep 19 12:59:33.486552 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486520    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-dshm\") pod \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") "
Sep 19 12:59:33.486652 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486551    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-home" (OuterVolumeSpecName: "home") pod "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" (UID: "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:59:33.486652 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486558    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tmp-dir\") pod \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\" (UID: \"3d9d0109-415e-4d7b-8bd7-d797c1dc4aff\") "
Sep 19 12:59:33.486763 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486718    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-model-cache" (OuterVolumeSpecName: "model-cache") pod "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" (UID: "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:59:33.486826 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486766    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" (UID: "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:59:33.486930 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486915    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:33.486989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486936    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:33.486989 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.486948    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:33.488798 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.488770    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-kube-api-access-xdw9l" (OuterVolumeSpecName: "kube-api-access-xdw9l") pod "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" (UID: "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff"). InnerVolumeSpecName "kube-api-access-xdw9l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 12:59:33.488931 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.488892    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-dshm" (OuterVolumeSpecName: "dshm") pod "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" (UID: "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 12:59:33.488994 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.488946    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" (UID: "3d9d0109-415e-4d7b-8bd7-d797c1dc4aff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 12:59:33.587550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.587517    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:33.587550 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.587548    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdw9l\" (UniqueName: \"kubernetes.io/projected/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-kube-api-access-xdw9l\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:33.587758 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.587559    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 12:59:33.724082 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.724049    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"]
Sep 19 12:59:33.726339 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:33.726311    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-f32129/ipp-route-f32129-kserve-85bf75dcb7-vd7bb"]
Sep 19 12:59:34.739702 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:34.739671    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" path="/var/lib/kubelet/pods/3d9d0109-415e-4d7b-8bd7-d797c1dc4aff/volumes"
Sep 19 12:59:54.866283 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:54.866217    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 12:59:55.900886 ip-10-0-131-60 kubenswrapper[2571]: I0919 12:59:55.900850    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:00.146692 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.146648    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"]
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.146992    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" containerName="payload-processing"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147003    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" containerName="payload-processing"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147012    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" containerName="main"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147017    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" containerName="main"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147032    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1db7d42d-d616-4fb9-8f25-a2bf8af580ca" containerName="revoke-keys"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147038    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="1db7d42d-d616-4fb9-8f25-a2bf8af580ca" containerName="revoke-keys"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147089    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="1db7d42d-d616-4fb9-8f25-a2bf8af580ca" containerName="revoke-keys"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147096    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d9d0109-415e-4d7b-8bd7-d797c1dc4aff" containerName="main"
Sep 19 13:00:00.147214 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.147102    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4db129a-b9cc-4df9-93a8-b7cc7a0880ea" containerName="payload-processing"
Sep 19 13:00:00.149933 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.149918    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"]
Sep 19 13:00:00.150052 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.150033    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"
Sep 19 13:00:00.152745 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.152724    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 13:00:00.153105 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.153091    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"
Sep 19 13:00:00.162509 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.162489    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"
Sep 19 13:00:00.162779 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.162752    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"]
Sep 19 13:00:00.165129 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.165067    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"]
Sep 19 13:00:00.171181 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.171155    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"
Sep 19 13:00:00.456865 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.456833    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"]
Sep 19 13:00:00.459907 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:00:00.459879    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd793ccf_3188_4a25_b45d_47210733472d.slice/crio-7cb5faf3426db17764e4abcc5a951c46fe9a362d36564d32b9b557bffbccebd9 WatchSource:0}: Error finding container 7cb5faf3426db17764e4abcc5a951c46fe9a362d36564d32b9b557bffbccebd9: Status 404 returned error can't find the container with id 7cb5faf3426db17764e4abcc5a951c46fe9a362d36564d32b9b557bffbccebd9
Sep 19 13:00:00.469545 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.469524    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"]
Sep 19 13:00:00.473446 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:00:00.473421    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6362f8c_3b7d_408e_84a4_c5d5fa5e9829.slice/crio-76b02616f1f2f5cf8b133467621443b5ae53bb6614d5cbba69c3aed0a27d4529 WatchSource:0}: Error finding container 76b02616f1f2f5cf8b133467621443b5ae53bb6614d5cbba69c3aed0a27d4529: Status 404 returned error can't find the container with id 76b02616f1f2f5cf8b133467621443b5ae53bb6614d5cbba69c3aed0a27d4529
Sep 19 13:00:00.525317 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.525286    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn" event={"ID":"d6362f8c-3b7d-408e-84a4-c5d5fa5e9829","Type":"ContainerStarted","Data":"76b02616f1f2f5cf8b133467621443b5ae53bb6614d5cbba69c3aed0a27d4529"}
Sep 19 13:00:00.526412 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:00.526391    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k" event={"ID":"cd793ccf-3188-4a25-b45d-47210733472d","Type":"ContainerStarted","Data":"7cb5faf3426db17764e4abcc5a951c46fe9a362d36564d32b9b557bffbccebd9"}
Sep 19 13:00:01.533129 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.533093    2571 generic.go:356] "Generic (PLEG): container finished" podID="d6362f8c-3b7d-408e-84a4-c5d5fa5e9829" containerID="0c7b62911b06f22e5c39b13fe634a26d3e72270b45c9cb39dd59b01e548bc0ea" exitCode=0
Sep 19 13:00:01.533561 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.533184    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn" event={"ID":"d6362f8c-3b7d-408e-84a4-c5d5fa5e9829","Type":"ContainerDied","Data":"0c7b62911b06f22e5c39b13fe634a26d3e72270b45c9cb39dd59b01e548bc0ea"}
Sep 19 13:00:01.534714 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.534687    2571 generic.go:356] "Generic (PLEG): container finished" podID="cd793ccf-3188-4a25-b45d-47210733472d" containerID="704c6b6bc0d646951f0bf66aeadcece4d97baf7f8db824153eadc0a2645da52d" exitCode=0
Sep 19 13:00:01.534844 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.534764    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k" event={"ID":"cd793ccf-3188-4a25-b45d-47210733472d","Type":"ContainerDied","Data":"704c6b6bc0d646951f0bf66aeadcece4d97baf7f8db824153eadc0a2645da52d"}
Sep 19 13:00:01.554269 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.554233    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:01.660788 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.660749    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:01.782805 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.782776    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"]
Sep 19 13:00:01.783024 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:01.782991    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy" containerID="cri-o://81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7" gracePeriod=30
Sep 19 13:00:02.063928 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:02.063845    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:02.601997 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:02.601965    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:02.694297 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:02.694270    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"
Sep 19 13:00:02.721192 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:02.721169    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"
Sep 19 13:00:03.546584 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.546554    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k"
Sep 19 13:00:03.546779 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.546608    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29830380-jlb4k" event={"ID":"cd793ccf-3188-4a25-b45d-47210733472d","Type":"ContainerDied","Data":"7cb5faf3426db17764e4abcc5a951c46fe9a362d36564d32b9b557bffbccebd9"}
Sep 19 13:00:03.546779 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.546672    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7cb5faf3426db17764e4abcc5a951c46fe9a362d36564d32b9b557bffbccebd9"
Sep 19 13:00:03.548008 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.547986    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn" event={"ID":"d6362f8c-3b7d-408e-84a4-c5d5fa5e9829","Type":"ContainerDied","Data":"76b02616f1f2f5cf8b133467621443b5ae53bb6614d5cbba69c3aed0a27d4529"}
Sep 19 13:00:03.548118 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.548010    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"
Sep 19 13:00:03.548118 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.548025    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="76b02616f1f2f5cf8b133467621443b5ae53bb6614d5cbba69c3aed0a27d4529"
Sep 19 13:00:03.622254 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:03.622220    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:04.384469 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:04.384441    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:05.326931 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:05.326895    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:06.016597 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:06.016562    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:07.024379 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.024356    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 13:00:07.070182 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070093    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-envoy\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070182 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070146    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-socket\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070182 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070171    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-certs\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070205    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrzmp\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-kube-api-access-lrzmp\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070239    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-istio-token\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070297    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-data\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070324    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6df700-301e-4da3-897e-0f8f196c8326-istio-podinfo\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070352    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6df700-301e-4da3-897e-0f8f196c8326-istiod-ca-cert\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070425    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:07.070781 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070473    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-credential-socket\") pod \"aa6df700-301e-4da3-897e-0f8f196c8326\" (UID: \"aa6df700-301e-4da3-897e-0f8f196c8326\") "
Sep 19 13:00:07.070781 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070532    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:07.070781 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070634    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-data" (OuterVolumeSpecName: "istio-data") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:07.070781 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070681    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:07.071022 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070851    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-credential-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.071022 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070870    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.071022 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070884    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-workload-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.071022 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070897    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-data\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.071022 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.070902    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa6df700-301e-4da3-897e-0f8f196c8326-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:07.072549 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.072516    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-istio-token" (OuterVolumeSpecName: "istio-token") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:07.072703 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.072561    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:07.072703 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.072570    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-kube-api-access-lrzmp" (OuterVolumeSpecName: "kube-api-access-lrzmp") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "kube-api-access-lrzmp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:07.072703 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.072609    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aa6df700-301e-4da3-897e-0f8f196c8326-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aa6df700-301e-4da3-897e-0f8f196c8326" (UID: "aa6df700-301e-4da3-897e-0f8f196c8326"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 13:00:07.105868 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.105841    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:07.165679 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.165628    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:07.171851 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.171829    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aa6df700-301e-4da3-897e-0f8f196c8326-istio-envoy\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.171851 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.171853    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrzmp\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-kube-api-access-lrzmp\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.171999 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.171863    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aa6df700-301e-4da3-897e-0f8f196c8326-istio-token\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.171999 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.171871    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aa6df700-301e-4da3-897e-0f8f196c8326-istio-podinfo\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.171999 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.171879    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aa6df700-301e-4da3-897e-0f8f196c8326-istiod-ca-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:07.569654 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.569619    2571 generic.go:356] "Generic (PLEG): container finished" podID="aa6df700-301e-4da3-897e-0f8f196c8326" containerID="81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7" exitCode=0
Sep 19 13:00:07.569811 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.569663    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"
Sep 19 13:00:07.569811 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.569673    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" event={"ID":"aa6df700-301e-4da3-897e-0f8f196c8326","Type":"ContainerDied","Data":"81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7"}
Sep 19 13:00:07.569811 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.569715    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x" event={"ID":"aa6df700-301e-4da3-897e-0f8f196c8326","Type":"ContainerDied","Data":"bb26d21dca97c8da5debe689e6e324159301de8ab79767b07c254f0c4de2fefd"}
Sep 19 13:00:07.569811 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.569736    2571 scope.go:117] "RemoveContainer" containerID="81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7"
Sep 19 13:00:07.578935 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.578917    2571 scope.go:117] "RemoveContainer" containerID="be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"
Sep 19 13:00:07.586673 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.586653    2571 scope.go:117] "RemoveContainer" containerID="81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7"
Sep 19 13:00:07.586933 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:00:07.586916    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7\": container with ID starting with 81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7 not found: ID does not exist" containerID="81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7"
Sep 19 13:00:07.586980 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.586940    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7"} err="failed to get container status \"81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7\": rpc error: code = NotFound desc = could not find container \"81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7\": container with ID starting with 81203d293533f0f0ee5e91a37933b9626aff901eb6280dfc1db0b3968c1b08d7 not found: ID does not exist"
Sep 19 13:00:07.586980 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.586955    2571 scope.go:117] "RemoveContainer" containerID="be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"
Sep 19 13:00:07.587211 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:00:07.587192    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4\": container with ID starting with be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4 not found: ID does not exist" containerID="be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"
Sep 19 13:00:07.587258 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.587218    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4"} err="failed to get container status \"be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4\": rpc error: code = NotFound desc = could not find container \"be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4\": container with ID starting with be2eca7f1b83de11bbeb4ed8a81657e1ff5c16465f6ef2ebaa5ed63a052ac1e4 not found: ID does not exist"
Sep 19 13:00:07.594780 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.594758    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"]
Sep 19 13:00:07.598401 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:07.598378    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-f32129-openshift-default-7698d45864-fvr8x"]
Sep 19 13:00:08.739916 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:08.739884    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" path="/var/lib/kubelet/pods/aa6df700-301e-4da3-897e-0f8f196c8326/volumes"
Sep 19 13:00:12.773992 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:12.773944    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:12.810119 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:12.809274    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"]
Sep 19 13:00:12.815195 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:12.811664    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8" podUID="cf0e09bc-5877-4cdc-86db-517086a9485d" containerName="payload-processing" containerID="cri-o://33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31" gracePeriod=30
Sep 19 13:00:12.871098 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:12.871060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:12.968577 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:12.968538    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:13.037105 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.037027    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:13.067176 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.067156    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 13:00:13.121544 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.121516    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf0e09bc-5877-4cdc-86db-517086a9485d-plugins-config-volume\") pod \"cf0e09bc-5877-4cdc-86db-517086a9485d\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") "
Sep 19 13:00:13.121708 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.121558    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4s24p\" (UniqueName: \"kubernetes.io/projected/cf0e09bc-5877-4cdc-86db-517086a9485d-kube-api-access-4s24p\") pod \"cf0e09bc-5877-4cdc-86db-517086a9485d\" (UID: \"cf0e09bc-5877-4cdc-86db-517086a9485d\") "
Sep 19 13:00:13.121864 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.121843    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf0e09bc-5877-4cdc-86db-517086a9485d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf0e09bc-5877-4cdc-86db-517086a9485d" (UID: "cf0e09bc-5877-4cdc-86db-517086a9485d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:13.123742 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.123715    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0e09bc-5877-4cdc-86db-517086a9485d-kube-api-access-4s24p" (OuterVolumeSpecName: "kube-api-access-4s24p") pod "cf0e09bc-5877-4cdc-86db-517086a9485d" (UID: "cf0e09bc-5877-4cdc-86db-517086a9485d"). InnerVolumeSpecName "kube-api-access-4s24p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:13.222805 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.222766    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf0e09bc-5877-4cdc-86db-517086a9485d-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:13.222805 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.222799    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s24p\" (UniqueName: \"kubernetes.io/projected/cf0e09bc-5877-4cdc-86db-517086a9485d-kube-api-access-4s24p\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:13.344802 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.344715    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:13.467476 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.467421    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:13.600928 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.600834    2571 generic.go:356] "Generic (PLEG): container finished" podID="cf0e09bc-5877-4cdc-86db-517086a9485d" containerID="33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31" exitCode=0
Sep 19 13:00:13.600928 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.600880    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"
Sep 19 13:00:13.600928 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.600918    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8" event={"ID":"cf0e09bc-5877-4cdc-86db-517086a9485d","Type":"ContainerDied","Data":"33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31"}
Sep 19 13:00:13.601177 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.600966    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8" event={"ID":"cf0e09bc-5877-4cdc-86db-517086a9485d","Type":"ContainerDied","Data":"a947aa6c1e094641b438f1ffa8e215ef5e28b82a9888375da9964e207b52bfd6"}
Sep 19 13:00:13.601177 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.600989    2571 scope.go:117] "RemoveContainer" containerID="33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31"
Sep 19 13:00:13.609271 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.609228    2571 scope.go:117] "RemoveContainer" containerID="33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31"
Sep 19 13:00:13.609559 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:00:13.609537    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31\": container with ID starting with 33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31 not found: ID does not exist" containerID="33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31"
Sep 19 13:00:13.609662 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.609567    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31"} err="failed to get container status \"33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31\": rpc error: code = NotFound desc = could not find container \"33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31\": container with ID starting with 33c98d90ef044e07e13c4b0d5b61debcf63dda59b58af12daeac1bbe663ecb31 not found: ID does not exist"
Sep 19 13:00:13.622322 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.622298    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"]
Sep 19 13:00:13.623914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:13.623896    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-4e754a-554f4c979f-ctqj8"]
Sep 19 13:00:14.204100 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:14.204060    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:14.739851 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:14.739816    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf0e09bc-5877-4cdc-86db-517086a9485d" path="/var/lib/kubelet/pods/cf0e09bc-5877-4cdc-86db-517086a9485d/volumes"
Sep 19 13:00:14.928473 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:14.928426    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:00:15.152826 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:15.152738    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:19.675965 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:19.675911    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:19.758966 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:19.758930    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:19.930320 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:19.930247    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"]
Sep 19 13:00:19.930487 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:19.930450    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy" containerID="cri-o://f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737" gracePeriod=30
Sep 19 13:00:19.960550 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:19.960468    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:20.285477 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285424    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"]
Sep 19 13:00:20.285753 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285741    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6362f8c-3b7d-408e-84a4-c5d5fa5e9829" containerName="cleanup"
Sep 19 13:00:20.285798 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285755    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6362f8c-3b7d-408e-84a4-c5d5fa5e9829" containerName="cleanup"
Sep 19 13:00:20.285798 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285765    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd793ccf-3188-4a25-b45d-47210733472d" containerName="cleanup"
Sep 19 13:00:20.285798 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285772    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd793ccf-3188-4a25-b45d-47210733472d" containerName="cleanup"
Sep 19 13:00:20.285798 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285787    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy"
Sep 19 13:00:20.285798 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285792    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285799    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0e09bc-5877-4cdc-86db-517086a9485d" containerName="payload-processing"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285805    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0e09bc-5877-4cdc-86db-517086a9485d" containerName="payload-processing"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285817    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285822    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285884    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285890    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0e09bc-5877-4cdc-86db-517086a9485d" containerName="payload-processing"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285900    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6362f8c-3b7d-408e-84a4-c5d5fa5e9829" containerName="cleanup"
Sep 19 13:00:20.285963 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.285906    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd793ccf-3188-4a25-b45d-47210733472d" containerName="cleanup"
Sep 19 13:00:20.290252 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.290234    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:20.293044 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.293022    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 13:00:20.293158 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.293022    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 13:00:20.298157 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.298076    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"]
Sep 19 13:00:20.383678 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.383634    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4d7de-db0f-4bcc-865f-de57ce354f44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg\" (UID: \"2ac4d7de-db0f-4bcc-865f-de57ce354f44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:20.485123 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.485088    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4d7de-db0f-4bcc-865f-de57ce354f44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg\" (UID: \"2ac4d7de-db0f-4bcc-865f-de57ce354f44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:20.485726 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.485708    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4d7de-db0f-4bcc-865f-de57ce354f44-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg\" (UID: \"2ac4d7de-db0f-4bcc-865f-de57ce354f44\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:20.601392 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.601311    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:20.871427 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:20.871405    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"]
Sep 19 13:00:20.873437 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:00:20.873411    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ac4d7de_db0f_4bcc_865f_de57ce354f44.slice/crio-0d22208c70c29d261a89bbfcaa01dba487c6d174c47acbf9b9f9f0416b34ddb9 WatchSource:0}: Error finding container 0d22208c70c29d261a89bbfcaa01dba487c6d174c47acbf9b9f9f0416b34ddb9: Status 404 returned error can't find the container with id 0d22208c70c29d261a89bbfcaa01dba487c6d174c47acbf9b9f9f0416b34ddb9
Sep 19 13:00:21.088624 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:21.088589    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:21.641769 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:21.641739    2571 generic.go:356] "Generic (PLEG): container finished" podID="2ac4d7de-db0f-4bcc-865f-de57ce354f44" containerID="6f1eda21a4ee7f6d795cd1e85ee92682f39a1ab063bb2c2f8b1da18e539624ad" exitCode=0
Sep 19 13:00:21.641940 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:21.641821    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg" event={"ID":"2ac4d7de-db0f-4bcc-865f-de57ce354f44","Type":"ContainerDied","Data":"6f1eda21a4ee7f6d795cd1e85ee92682f39a1ab063bb2c2f8b1da18e539624ad"}
Sep 19 13:00:21.641940 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:21.641873    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg" event={"ID":"2ac4d7de-db0f-4bcc-865f-de57ce354f44","Type":"ContainerStarted","Data":"0d22208c70c29d261a89bbfcaa01dba487c6d174c47acbf9b9f9f0416b34ddb9"}
Sep 19 13:00:22.035689 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:22.035654    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:22.735779 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:22.735739    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:22.772698 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:22.772675    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:22.806081 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:22.806051    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4d7de-db0f-4bcc-865f-de57ce354f44-maas-api-service-ca\") pod \"2ac4d7de-db0f-4bcc-865f-de57ce354f44\" (UID: \"2ac4d7de-db0f-4bcc-865f-de57ce354f44\") "
Sep 19 13:00:22.806484 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:22.806448    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2ac4d7de-db0f-4bcc-865f-de57ce354f44-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2ac4d7de-db0f-4bcc-865f-de57ce354f44" (UID: "2ac4d7de-db0f-4bcc-865f-de57ce354f44"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:22.907088 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:22.907006    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ac4d7de-db0f-4bcc-865f-de57ce354f44-maas-api-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:23.652143 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:23.652107    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg" event={"ID":"2ac4d7de-db0f-4bcc-865f-de57ce354f44","Type":"ContainerDied","Data":"0d22208c70c29d261a89bbfcaa01dba487c6d174c47acbf9b9f9f0416b34ddb9"}
Sep 19 13:00:23.652143 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:23.652143    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"
Sep 19 13:00:23.652607 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:23.652181    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d22208c70c29d261a89bbfcaa01dba487c6d174c47acbf9b9f9f0416b34ddb9"
Sep 19 13:00:25.177230 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.177206    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 13:00:25.226121 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226089    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-token\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226284 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226167    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-data\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226352 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226281    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-certs\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226352 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226321    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dd99\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-kube-api-access-5dd99\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226445 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226356    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-credential-socket\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226445 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226376    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-socket\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226445 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226394    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72dd51a5-5996-484c-8843-39d4606d2ba3-istiod-ca-cert\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226636 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226449    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-envoy\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226636 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226478    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-data" (OuterVolumeSpecName: "istio-data") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:25.226636 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226497    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-podinfo\") pod \"72dd51a5-5996-484c-8843-39d4606d2ba3\" (UID: \"72dd51a5-5996-484c-8843-39d4606d2ba3\") "
Sep 19 13:00:25.226636 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226585    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:25.226838 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226660    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:25.226838 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226820    2571 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-data\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.226838 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226837    2571 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.227000 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226852    2571 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-credential-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.227000 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226886    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:25.227000 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.226978    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72dd51a5-5996-484c-8843-39d4606d2ba3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:25.228689 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.228663    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:00:25.228919 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.228885    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 19 13:00:25.229009 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.228983    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-token" (OuterVolumeSpecName: "istio-token") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:25.229059 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.228984    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-kube-api-access-5dd99" (OuterVolumeSpecName: "kube-api-access-5dd99") pod "72dd51a5-5996-484c-8843-39d4606d2ba3" (UID: "72dd51a5-5996-484c-8843-39d4606d2ba3"). InnerVolumeSpecName "kube-api-access-5dd99". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:25.328279 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.328233    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dd99\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-kube-api-access-5dd99\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.328279 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.328269    2571 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-workload-socket\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.328279 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.328284    2571 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72dd51a5-5996-484c-8843-39d4606d2ba3-istiod-ca-cert\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.328576 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.328298    2571 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-envoy\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.328576 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.328309    2571 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-podinfo\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.328576 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.328320    2571 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72dd51a5-5996-484c-8843-39d4606d2ba3-istio-token\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:25.663645 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.663557    2571 generic.go:356] "Generic (PLEG): container finished" podID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerID="f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737" exitCode=0
Sep 19 13:00:25.663645 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.663604    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"
Sep 19 13:00:25.663826 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.663652    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" event={"ID":"72dd51a5-5996-484c-8843-39d4606d2ba3","Type":"ContainerDied","Data":"f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737"}
Sep 19 13:00:25.663826 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.663706    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq" event={"ID":"72dd51a5-5996-484c-8843-39d4606d2ba3","Type":"ContainerDied","Data":"25ee538e065645650325fa8ec9d41178edfad89afbf57c21d3064315c5fe8dcb"}
Sep 19 13:00:25.663826 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.663731    2571 scope.go:117] "RemoveContainer" containerID="f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737"
Sep 19 13:00:25.674813 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.674794    2571 scope.go:117] "RemoveContainer" containerID="3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"
Sep 19 13:00:25.684242 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.684211    2571 scope.go:117] "RemoveContainer" containerID="f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737"
Sep 19 13:00:25.684558 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:00:25.684529    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737\": container with ID starting with f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737 not found: ID does not exist" containerID="f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737"
Sep 19 13:00:25.684642 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.684569    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737"} err="failed to get container status \"f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737\": rpc error: code = NotFound desc = could not find container \"f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737\": container with ID starting with f0bbb0032ee3a1e522e4233383a085bdbf8ef2c0fa3eb323a6b308fb0a125737 not found: ID does not exist"
Sep 19 13:00:25.684642 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.684592    2571 scope.go:117] "RemoveContainer" containerID="3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"
Sep 19 13:00:25.684875 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:00:25.684856    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b\": container with ID starting with 3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b not found: ID does not exist" containerID="3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"
Sep 19 13:00:25.684934 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.684881    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b"} err="failed to get container status \"3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b\": rpc error: code = NotFound desc = could not find container \"3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b\": container with ID starting with 3266adbddec2128c7059e02be5179bb241205e263f85865aaab7123ea5bddb9b not found: ID does not exist"
Sep 19 13:00:25.688454 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.688434    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"]
Sep 19 13:00:25.692349 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:25.692327    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-4e754a-openshift-default-666c4f55b-xxdcq"]
Sep 19 13:00:26.740026 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:26.739994    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" path="/var/lib/kubelet/pods/72dd51a5-5996-484c-8843-39d4606d2ba3/volumes"
Sep 19 13:00:30.808654 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:30.808614    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:30.812411 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:30.812387    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"]
Sep 19 13:00:30.815571 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:30.815548    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-b0e30e-29830380-tncxn"]
Sep 19 13:00:30.878611 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:30.878571    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:30.947217 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:30.947188    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:31.009059 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:31.009034    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"]
Sep 19 13:00:31.011416 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:31.011397    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-b0e30e-hrppg"]
Sep 19 13:00:31.018537 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:31.018511    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:31.405776 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:31.405744    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:31.505473 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:31.505425    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:32.425017 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:32.424989    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:32.739257 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:32.739230    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ac4d7de-db0f-4bcc-865f-de57ce354f44" path="/var/lib/kubelet/pods/2ac4d7de-db0f-4bcc-865f-de57ce354f44/volumes"
Sep 19 13:00:32.739527 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:32.739514    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6362f8c-3b7d-408e-84a4-c5d5fa5e9829" path="/var/lib/kubelet/pods/d6362f8c-3b7d-408e-84a4-c5d5fa5e9829/volumes"
Sep 19 13:00:33.838551 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:33.838521    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:34.042642 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:34.042607    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:00:37.629128 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:37.629090    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:38.208674 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208639    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"]
Sep 19 13:00:38.208946 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208933    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy"
Sep 19 13:00:38.208998 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208947    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy"
Sep 19 13:00:38.208998 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208959    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy"
Sep 19 13:00:38.208998 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208966    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy"
Sep 19 13:00:38.208998 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208976    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ac4d7de-db0f-4bcc-865f-de57ce354f44" containerName="revoke-keys"
Sep 19 13:00:38.208998 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.208983    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ac4d7de-db0f-4bcc-865f-de57ce354f44" containerName="revoke-keys"
Sep 19 13:00:38.209174 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.209034    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa6df700-301e-4da3-897e-0f8f196c8326" containerName="istio-proxy"
Sep 19 13:00:38.209174 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.209043    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ac4d7de-db0f-4bcc-865f-de57ce354f44" containerName="revoke-keys"
Sep 19 13:00:38.209174 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.209050    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy"
Sep 19 13:00:38.209174 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.209056    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="72dd51a5-5996-484c-8843-39d4606d2ba3" containerName="istio-proxy"
Sep 19 13:00:38.213594 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.213578    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:38.216342 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.216321    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5vkdc\""
Sep 19 13:00:38.216491 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.216430    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 13:00:38.220113 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.220087    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"]
Sep 19 13:00:38.342879 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.342841    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc90e177-b7ca-4456-a409-e7c2309de128-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng\" (UID: \"fc90e177-b7ca-4456-a409-e7c2309de128\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:38.443295 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.443263    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc90e177-b7ca-4456-a409-e7c2309de128-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng\" (UID: \"fc90e177-b7ca-4456-a409-e7c2309de128\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:38.443855 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.443836    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc90e177-b7ca-4456-a409-e7c2309de128-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng\" (UID: \"fc90e177-b7ca-4456-a409-e7c2309de128\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:38.526913 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.526826    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:38.796477 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:38.796438    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"]
Sep 19 13:00:38.798684 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:00:38.798644    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc90e177_b7ca_4456_a409_e7c2309de128.slice/crio-ba24786fe5a6c9cfb0a1e7bc2edaad2b8d6ba5ba9d0ccf1ab039829850413a03 WatchSource:0}: Error finding container ba24786fe5a6c9cfb0a1e7bc2edaad2b8d6ba5ba9d0ccf1ab039829850413a03: Status 404 returned error can't find the container with id ba24786fe5a6c9cfb0a1e7bc2edaad2b8d6ba5ba9d0ccf1ab039829850413a03
Sep 19 13:00:39.334918 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:39.334875    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:39.730772 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:39.730743    2571 generic.go:356] "Generic (PLEG): container finished" podID="fc90e177-b7ca-4456-a409-e7c2309de128" containerID="40cbf961a94758e152f070cab653f2bff90e09c7e413d29c69d9f0288d288b17" exitCode=0
Sep 19 13:00:39.730955 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:39.730821    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng" event={"ID":"fc90e177-b7ca-4456-a409-e7c2309de128","Type":"ContainerDied","Data":"40cbf961a94758e152f070cab653f2bff90e09c7e413d29c69d9f0288d288b17"}
Sep 19 13:00:39.730955 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:39.730862    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng" event={"ID":"fc90e177-b7ca-4456-a409-e7c2309de128","Type":"ContainerStarted","Data":"ba24786fe5a6c9cfb0a1e7bc2edaad2b8d6ba5ba9d0ccf1ab039829850413a03"}
Sep 19 13:00:40.072770 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:40.072669    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:40.859511 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:40.859488    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:40.965964 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:40.965931    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc90e177-b7ca-4456-a409-e7c2309de128-maas-api-service-ca\") pod \"fc90e177-b7ca-4456-a409-e7c2309de128\" (UID: \"fc90e177-b7ca-4456-a409-e7c2309de128\") "
Sep 19 13:00:40.966334 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:40.966312    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc90e177-b7ca-4456-a409-e7c2309de128-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc90e177-b7ca-4456-a409-e7c2309de128" (UID: "fc90e177-b7ca-4456-a409-e7c2309de128"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:41.067077 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:41.066989    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc90e177-b7ca-4456-a409-e7c2309de128-maas-api-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:41.742015 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:41.741985    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"
Sep 19 13:00:41.742577 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:41.742029    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng" event={"ID":"fc90e177-b7ca-4456-a409-e7c2309de128","Type":"ContainerDied","Data":"ba24786fe5a6c9cfb0a1e7bc2edaad2b8d6ba5ba9d0ccf1ab039829850413a03"}
Sep 19 13:00:41.742577 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:41.742070    2571 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ba24786fe5a6c9cfb0a1e7bc2edaad2b8d6ba5ba9d0ccf1ab039829850413a03"
Sep 19 13:00:48.781487 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.781436    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:48.811507 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.811451    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"]
Sep 19 13:00:48.811743 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.811718    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr" podUID="95630877-88d2-4291-a310-2c1054378869" containerName="payload-pre-processing" containerID="cri-o://ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798" gracePeriod=30
Sep 19 13:00:48.874655 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.874620    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:48.977148 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.977115    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"]
Sep 19 13:00:48.980293 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.980262    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:48.981763 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:48.981736    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-5d6124-6cgng"]
Sep 19 13:00:49.066386 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.066360    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 13:00:49.077067 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.077032    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:49.238251 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.238216    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r95dn\" (UniqueName: \"kubernetes.io/projected/95630877-88d2-4291-a310-2c1054378869-kube-api-access-r95dn\") pod \"95630877-88d2-4291-a310-2c1054378869\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") "
Sep 19 13:00:49.238425 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.238279    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95630877-88d2-4291-a310-2c1054378869-plugins-config-volume\") pod \"95630877-88d2-4291-a310-2c1054378869\" (UID: \"95630877-88d2-4291-a310-2c1054378869\") "
Sep 19 13:00:49.238622 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.238601    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95630877-88d2-4291-a310-2c1054378869-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "95630877-88d2-4291-a310-2c1054378869" (UID: "95630877-88d2-4291-a310-2c1054378869"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:00:49.240357 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.240331    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95630877-88d2-4291-a310-2c1054378869-kube-api-access-r95dn" (OuterVolumeSpecName: "kube-api-access-r95dn") pod "95630877-88d2-4291-a310-2c1054378869" (UID: "95630877-88d2-4291-a310-2c1054378869"). InnerVolumeSpecName "kube-api-access-r95dn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:00:49.339577 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.339498    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r95dn\" (UniqueName: \"kubernetes.io/projected/95630877-88d2-4291-a310-2c1054378869-kube-api-access-r95dn\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:49.339577 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.339531    2571 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95630877-88d2-4291-a310-2c1054378869-plugins-config-volume\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:00:49.354644 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.354611    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:49.454152 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.454106    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:49.787183 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.787154    2571 generic.go:356] "Generic (PLEG): container finished" podID="95630877-88d2-4291-a310-2c1054378869" containerID="ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798" exitCode=0
Sep 19 13:00:49.787621 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.787196    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"
Sep 19 13:00:49.787621 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.787255    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr" event={"ID":"95630877-88d2-4291-a310-2c1054378869","Type":"ContainerDied","Data":"ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798"}
Sep 19 13:00:49.787621 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.787305    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr" event={"ID":"95630877-88d2-4291-a310-2c1054378869","Type":"ContainerDied","Data":"3bf31addfd6837a78ccee2598fec5e8ebb89a5230ebd9f01b5a37d8d047c91d0"}
Sep 19 13:00:49.787621 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.787326    2571 scope.go:117] "RemoveContainer" containerID="ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798"
Sep 19 13:00:49.796148 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.796132    2571 scope.go:117] "RemoveContainer" containerID="ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798"
Sep 19 13:00:49.796415 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:00:49.796397    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798\": container with ID starting with ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798 not found: ID does not exist" containerID="ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798"
Sep 19 13:00:49.796597 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.796423    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798"} err="failed to get container status \"ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798\": rpc error: code = NotFound desc = could not find container \"ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798\": container with ID starting with ee0facc4b1b986c42e82444cfa1a1a1b9d2dc538a7a2f7302c6d20732f9f9798 not found: ID does not exist"
Sep 19 13:00:49.809178 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.809156    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"]
Sep 19 13:00:49.822328 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:49.822298    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w3-5d6124-5f7d79dcc8-4s7wr"]
Sep 19 13:00:50.739377 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:50.739345    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95630877-88d2-4291-a310-2c1054378869" path="/var/lib/kubelet/pods/95630877-88d2-4291-a310-2c1054378869/volumes"
Sep 19 13:00:50.739729 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:50.739715    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc90e177-b7ca-4456-a409-e7c2309de128" path="/var/lib/kubelet/pods/fc90e177-b7ca-4456-a409-e7c2309de128/volumes"
Sep 19 13:00:51.233401 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:51.233367    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:00:55.202434 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:55.202386    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:55.276472 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:55.276438    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:00:55.488570 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:00:55.488534    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:01:00.038345 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:01:00.038270    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:02:20.082951 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:02:20.082908    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:02:20.241998 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:02:20.241964    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:02:21.741074 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:02:21.741031    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:02:36.681642 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:02:36.681539    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:02:59.471783 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:02:59.471748    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:03:06.030177 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:03:06.030135    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:03:13.763766 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:03:13.763729    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:03:22.081533 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:03:22.081503    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:03:33.332634 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:03:33.332595    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:03:40.231426 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:03:40.231392    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:04:37.481756 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:04:37.481665    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:05:07.823026 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:07.822983    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:05:14.536351 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:14.536316    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:05:19.091667 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:19.091630    2571 scope.go:117] "RemoveContainer" containerID="b85d373d35be0d832a425b60ddeabc1ed2c4b104ce825a37721c84eeb193cca1"
Sep 19 13:05:24.031630 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:24.031597    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:05:32.228496 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228410    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"]
Sep 19 13:05:32.228914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228770    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc90e177-b7ca-4456-a409-e7c2309de128" containerName="revoke-keys"
Sep 19 13:05:32.228914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228784    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc90e177-b7ca-4456-a409-e7c2309de128" containerName="revoke-keys"
Sep 19 13:05:32.228914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228793    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95630877-88d2-4291-a310-2c1054378869" containerName="payload-pre-processing"
Sep 19 13:05:32.228914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228799    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="95630877-88d2-4291-a310-2c1054378869" containerName="payload-pre-processing"
Sep 19 13:05:32.228914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228861    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc90e177-b7ca-4456-a409-e7c2309de128" containerName="revoke-keys"
Sep 19 13:05:32.228914 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.228872    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="95630877-88d2-4291-a310-2c1054378869" containerName="payload-pre-processing"
Sep 19 13:05:32.233204 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.233187    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.236395 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.236375    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-8dc88c8f-kserve-self-signed-certs\""
Sep 19 13:05:32.242183 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.242162    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"]
Sep 19 13:05:32.263649 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.263625    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:05:32.347305 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.347284    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tmp-dir\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.347392 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.347308    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tls-certs\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.347392 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.347337    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-dshm\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.347505 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.347402    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-home\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.347505 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.347454    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5lzh\" (UniqueName: \"kubernetes.io/projected/e7ed595f-9a53-4e9a-9a5b-542be95f5725-kube-api-access-v5lzh\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.347583 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.347514    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-model-cache\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448517 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448494    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-dshm\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448612 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448534    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-home\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448612 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448566    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5lzh\" (UniqueName: \"kubernetes.io/projected/e7ed595f-9a53-4e9a-9a5b-542be95f5725-kube-api-access-v5lzh\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448612 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448600    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-model-cache\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448781 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448648    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tmp-dir\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448781 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448671    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tls-certs\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.448933 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.448915    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-home\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.449037 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.449011    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-model-cache\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.449147 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.449090    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tmp-dir\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.450829 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.450806    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-dshm\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.451009 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.450992    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tls-certs\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.457281 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.457254    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5lzh\" (UniqueName: \"kubernetes.io/projected/e7ed595f-9a53-4e9a-9a5b-542be95f5725-kube-api-access-v5lzh\") pod \"e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") " pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.542313 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.542255    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:32.816286 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:32.816261    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"]
Sep 19 13:05:32.818299 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:05:32.818269    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode7ed595f_9a53_4e9a_9a5b_542be95f5725.slice/crio-acca1de298ba265f39f60a17518ff8653ddddcd7402650a0bce90a48b49ab018 WatchSource:0}: Error finding container acca1de298ba265f39f60a17518ff8653ddddcd7402650a0bce90a48b49ab018: Status 404 returned error can't find the container with id acca1de298ba265f39f60a17518ff8653ddddcd7402650a0bce90a48b49ab018
Sep 19 13:05:33.097714 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:33.097635    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt" event={"ID":"e7ed595f-9a53-4e9a-9a5b-542be95f5725","Type":"ContainerStarted","Data":"e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1"}
Sep 19 13:05:33.097714 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:33.097680    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt" event={"ID":"e7ed595f-9a53-4e9a-9a5b-542be95f5725","Type":"ContainerStarted","Data":"acca1de298ba265f39f60a17518ff8653ddddcd7402650a0bce90a48b49ab018"}
Sep 19 13:05:33.116654 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:33.116614    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt" podStartSLOduration=1.116599481 podStartE2EDuration="1.116599481s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 13:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 13:05:33.116210646 +0000 UTC m=+2416.476453586" watchObservedRunningTime="2026-09-19 13:05:33.116599481 +0000 UTC m=+2416.476842417"
Sep 19 13:05:34.882410 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:34.882380    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:05:35.026585 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:35.026551    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:05:42.543380 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:42.543339    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:42.543919 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:42.543396    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:42.555656 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:42.555633    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:43.157223 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:43.157195    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:05:59.439233 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.439199    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"]
Sep 19 13:05:59.442292 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.442276    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.445221 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.445199    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-8dc88c8f-kserve-self-signed-certs\""
Sep 19 13:05:59.451014 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.450989    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"]
Sep 19 13:05:59.492930 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.492904    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-dshm\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.493060 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.492955    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-home\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.493060 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.493004    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-model-cache\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.493189 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.493065    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p6d7\" (UniqueName: \"kubernetes.io/projected/9207e8cb-1c34-4477-bc65-5d60f024973d-kube-api-access-8p6d7\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.493189 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.493149    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-tmp-dir\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.493315 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.493208    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9207e8cb-1c34-4477-bc65-5d60f024973d-tls-certs\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.544858 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.544829    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:05:59.594123 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594098    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9207e8cb-1c34-4477-bc65-5d60f024973d-tls-certs\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594226 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594137    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-dshm\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594226 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594158    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-home\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594226 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594179    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-model-cache\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594226 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594209    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p6d7\" (UniqueName: \"kubernetes.io/projected/9207e8cb-1c34-4477-bc65-5d60f024973d-kube-api-access-8p6d7\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594426 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594236    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-tmp-dir\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594660 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594628    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-home\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594758 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594660    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-model-cache\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.594758 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.594722    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-tmp-dir\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.596905 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.596886    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-dshm\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.597141 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.597123    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9207e8cb-1c34-4477-bc65-5d60f024973d-tls-certs\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.602116 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.602095    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p6d7\" (UniqueName: \"kubernetes.io/projected/9207e8cb-1c34-4477-bc65-5d60f024973d-kube-api-access-8p6d7\") pod \"e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") " pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:05:59.752444 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:05:59.752418    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:00.023590 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:00.023569    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"]
Sep 19 13:06:00.025753 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:06:00.025727    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9207e8cb_1c34_4477_bc65_5d60f024973d.slice/crio-ba17174bc9e4a50fe2340ce19827b01692284aff7ba8dce8dae583f8740b3eef WatchSource:0}: Error finding container ba17174bc9e4a50fe2340ce19827b01692284aff7ba8dce8dae583f8740b3eef: Status 404 returned error can't find the container with id ba17174bc9e4a50fe2340ce19827b01692284aff7ba8dce8dae583f8740b3eef
Sep 19 13:06:00.223779 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:00.223747    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf" event={"ID":"9207e8cb-1c34-4477-bc65-5d60f024973d","Type":"ContainerStarted","Data":"df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d"}
Sep 19 13:06:00.223779 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:00.223784    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf" event={"ID":"9207e8cb-1c34-4477-bc65-5d60f024973d","Type":"ContainerStarted","Data":"ba17174bc9e4a50fe2340ce19827b01692284aff7ba8dce8dae583f8740b3eef"}
Sep 19 13:06:00.241235 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:00.241193    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf" podStartSLOduration=1.241179473 podStartE2EDuration="1.241179473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 13:05:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 13:06:00.240029922 +0000 UTC m=+2443.600272865" watchObservedRunningTime="2026-09-19 13:06:00.241179473 +0000 UTC m=+2443.601422413"
Sep 19 13:06:02.442049 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:02.442012    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:06:09.752647 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:09.752604    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:09.753218 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:09.752672    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:09.765080 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:09.765053    2571 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:10.279289 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:10.279261    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:14.033413 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:14.033377    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"]
Sep 19 13:06:14.062379 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:14.062342    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:06:15.853626 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:15.853591    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"]
Sep 19 13:06:15.876988 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:15.876938    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:06:16.229561 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:16.229521    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:06:19.110190 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:19.110155    2571 scope.go:117] "RemoveContainer" containerID="0c7b62911b06f22e5c39b13fe634a26d3e72270b45c9cb39dd59b01e548bc0ea"
Sep 19 13:06:25.224018 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:25.223979    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:06:29.051971 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.051922    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt" podUID="e7ed595f-9a53-4e9a-9a5b-542be95f5725" containerName="main" containerID="cri-o://e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1" gracePeriod=45
Sep 19 13:06:29.290989 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.290968    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:06:29.355786 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.355706    2571 generic.go:356] "Generic (PLEG): container finished" podID="e7ed595f-9a53-4e9a-9a5b-542be95f5725" containerID="e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1" exitCode=0
Sep 19 13:06:29.355786 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.355778    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt" event={"ID":"e7ed595f-9a53-4e9a-9a5b-542be95f5725","Type":"ContainerDied","Data":"e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1"}
Sep 19 13:06:29.355980 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.355795    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"
Sep 19 13:06:29.355980 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.355812    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt" event={"ID":"e7ed595f-9a53-4e9a-9a5b-542be95f5725","Type":"ContainerDied","Data":"acca1de298ba265f39f60a17518ff8653ddddcd7402650a0bce90a48b49ab018"}
Sep 19 13:06:29.355980 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.355830    2571 scope.go:117] "RemoveContainer" containerID="e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1"
Sep 19 13:06:29.363610 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.363594    2571 scope.go:117] "RemoveContainer" containerID="e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1"
Sep 19 13:06:29.363851 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:06:29.363833    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1\": container with ID starting with e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1 not found: ID does not exist" containerID="e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1"
Sep 19 13:06:29.363911 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.363861    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1"} err="failed to get container status \"e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1\": rpc error: code = NotFound desc = could not find container \"e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1\": container with ID starting with e694ee57708ecea57230bf64d93cffe9608aea18812ccf989a52935f2208acf1 not found: ID does not exist"
Sep 19 13:06:29.442821 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.442800    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-model-cache\") pod \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") "
Sep 19 13:06:29.442902 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.442831    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-dshm\") pod \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") "
Sep 19 13:06:29.442902 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.442854    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5lzh\" (UniqueName: \"kubernetes.io/projected/e7ed595f-9a53-4e9a-9a5b-542be95f5725-kube-api-access-v5lzh\") pod \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") "
Sep 19 13:06:29.442902 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.442892    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tls-certs\") pod \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") "
Sep 19 13:06:29.443026 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.442908    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-home\") pod \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") "
Sep 19 13:06:29.443026 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.442962    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tmp-dir\") pod \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\" (UID: \"e7ed595f-9a53-4e9a-9a5b-542be95f5725\") "
Sep 19 13:06:29.443141 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.443067    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-model-cache" (OuterVolumeSpecName: "model-cache") pod "e7ed595f-9a53-4e9a-9a5b-542be95f5725" (UID: "e7ed595f-9a53-4e9a-9a5b-542be95f5725"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:29.443193 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.443170    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:29.443264 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.443235    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-home" (OuterVolumeSpecName: "home") pod "e7ed595f-9a53-4e9a-9a5b-542be95f5725" (UID: "e7ed595f-9a53-4e9a-9a5b-542be95f5725"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:29.443311 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.443262    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e7ed595f-9a53-4e9a-9a5b-542be95f5725" (UID: "e7ed595f-9a53-4e9a-9a5b-542be95f5725"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:29.444882 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.444863    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-dshm" (OuterVolumeSpecName: "dshm") pod "e7ed595f-9a53-4e9a-9a5b-542be95f5725" (UID: "e7ed595f-9a53-4e9a-9a5b-542be95f5725"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:29.445041 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.445019    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e7ed595f-9a53-4e9a-9a5b-542be95f5725" (UID: "e7ed595f-9a53-4e9a-9a5b-542be95f5725"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 13:06:29.445135 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.445066    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7ed595f-9a53-4e9a-9a5b-542be95f5725-kube-api-access-v5lzh" (OuterVolumeSpecName: "kube-api-access-v5lzh") pod "e7ed595f-9a53-4e9a-9a5b-542be95f5725" (UID: "e7ed595f-9a53-4e9a-9a5b-542be95f5725"). InnerVolumeSpecName "kube-api-access-v5lzh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:06:29.543488 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.543438    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:29.543488 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.543485    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:29.543488 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.543495    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5lzh\" (UniqueName: \"kubernetes.io/projected/e7ed595f-9a53-4e9a-9a5b-542be95f5725-kube-api-access-v5lzh\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:29.543668 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.543506    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e7ed595f-9a53-4e9a-9a5b-542be95f5725-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:29.543668 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.543514    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e7ed595f-9a53-4e9a-9a5b-542be95f5725-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:29.678332 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.678307    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"]
Sep 19 13:06:29.683579 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:29.683559    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-8dc88c8f-kserve-5b9969595-sm9dt"]
Sep 19 13:06:30.739082 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:30.739050    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7ed595f-9a53-4e9a-9a5b-542be95f5725" path="/var/lib/kubelet/pods/e7ed595f-9a53-4e9a-9a5b-542be95f5725/volumes"
Sep 19 13:06:30.872068 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:30.872028    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf" podUID="9207e8cb-1c34-4477-bc65-5d60f024973d" containerName="main" containerID="cri-o://df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d" gracePeriod=45
Sep 19 13:06:31.116524 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.116493    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:31.260837 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.260752    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-model-cache\") pod \"9207e8cb-1c34-4477-bc65-5d60f024973d\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") "
Sep 19 13:06:31.260837 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.260794    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-tmp-dir\") pod \"9207e8cb-1c34-4477-bc65-5d60f024973d\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") "
Sep 19 13:06:31.260837 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.260828    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8p6d7\" (UniqueName: \"kubernetes.io/projected/9207e8cb-1c34-4477-bc65-5d60f024973d-kube-api-access-8p6d7\") pod \"9207e8cb-1c34-4477-bc65-5d60f024973d\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") "
Sep 19 13:06:31.261131 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.260878    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-home\") pod \"9207e8cb-1c34-4477-bc65-5d60f024973d\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") "
Sep 19 13:06:31.261131 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.260926    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9207e8cb-1c34-4477-bc65-5d60f024973d-tls-certs\") pod \"9207e8cb-1c34-4477-bc65-5d60f024973d\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") "
Sep 19 13:06:31.261131 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.260975    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-dshm\") pod \"9207e8cb-1c34-4477-bc65-5d60f024973d\" (UID: \"9207e8cb-1c34-4477-bc65-5d60f024973d\") "
Sep 19 13:06:31.261131 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.261063    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-model-cache" (OuterVolumeSpecName: "model-cache") pod "9207e8cb-1c34-4477-bc65-5d60f024973d" (UID: "9207e8cb-1c34-4477-bc65-5d60f024973d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:31.261131 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.261082    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9207e8cb-1c34-4477-bc65-5d60f024973d" (UID: "9207e8cb-1c34-4477-bc65-5d60f024973d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:31.261363 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.261164    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-home" (OuterVolumeSpecName: "home") pod "9207e8cb-1c34-4477-bc65-5d60f024973d" (UID: "9207e8cb-1c34-4477-bc65-5d60f024973d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:31.261363 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.261306    2571 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-model-cache\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:31.261363 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.261324    2571 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-tmp-dir\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:31.261363 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.261333    2571 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-home\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:31.263182 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.263161    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-dshm" (OuterVolumeSpecName: "dshm") pod "9207e8cb-1c34-4477-bc65-5d60f024973d" (UID: "9207e8cb-1c34-4477-bc65-5d60f024973d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 19 13:06:31.263182 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.263168    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9207e8cb-1c34-4477-bc65-5d60f024973d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9207e8cb-1c34-4477-bc65-5d60f024973d" (UID: "9207e8cb-1c34-4477-bc65-5d60f024973d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 13:06:31.263339 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.263204    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9207e8cb-1c34-4477-bc65-5d60f024973d-kube-api-access-8p6d7" (OuterVolumeSpecName: "kube-api-access-8p6d7") pod "9207e8cb-1c34-4477-bc65-5d60f024973d" (UID: "9207e8cb-1c34-4477-bc65-5d60f024973d"). InnerVolumeSpecName "kube-api-access-8p6d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:06:31.362528 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.362493    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8p6d7\" (UniqueName: \"kubernetes.io/projected/9207e8cb-1c34-4477-bc65-5d60f024973d-kube-api-access-8p6d7\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:31.362528 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.362520    2571 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9207e8cb-1c34-4477-bc65-5d60f024973d-tls-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:31.362528 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.362531    2571 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9207e8cb-1c34-4477-bc65-5d60f024973d-dshm\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:31.368437 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.368408    2571 generic.go:356] "Generic (PLEG): container finished" podID="9207e8cb-1c34-4477-bc65-5d60f024973d" containerID="df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d" exitCode=0
Sep 19 13:06:31.368592 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.368475    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"
Sep 19 13:06:31.368592 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.368498    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf" event={"ID":"9207e8cb-1c34-4477-bc65-5d60f024973d","Type":"ContainerDied","Data":"df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d"}
Sep 19 13:06:31.368592 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.368545    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf" event={"ID":"9207e8cb-1c34-4477-bc65-5d60f024973d","Type":"ContainerDied","Data":"ba17174bc9e4a50fe2340ce19827b01692284aff7ba8dce8dae583f8740b3eef"}
Sep 19 13:06:31.368592 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.368566    2571 scope.go:117] "RemoveContainer" containerID="df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d"
Sep 19 13:06:31.377189 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.377168    2571 scope.go:117] "RemoveContainer" containerID="df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d"
Sep 19 13:06:31.377482 ip-10-0-131-60 kubenswrapper[2571]: E0919 13:06:31.377448    2571 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d\": container with ID starting with df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d not found: ID does not exist" containerID="df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d"
Sep 19 13:06:31.377549 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.377491    2571 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d"} err="failed to get container status \"df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d\": rpc error: code = NotFound desc = could not find container \"df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d\": container with ID starting with df7d2b8b508c4f2f3f72f93d52d50c4f23e4ef2b023b2594be7cd97f471cb63d not found: ID does not exist"
Sep 19 13:06:31.389014 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.388993    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"]
Sep 19 13:06:31.392251 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:31.392232    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-8dc88c8f-kserve-dcc6c8787-m6zpf"]
Sep 19 13:06:32.739029 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:32.738998    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9207e8cb-1c34-4477-bc65-5d60f024973d" path="/var/lib/kubelet/pods/9207e8cb-1c34-4477-bc65-5d60f024973d/volumes"
Sep 19 13:06:34.532748 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.532701    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:06:34.742412 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742383    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"]
Sep 19 13:06:34.742694 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742682    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7ed595f-9a53-4e9a-9a5b-542be95f5725" containerName="main"
Sep 19 13:06:34.742739 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742696    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7ed595f-9a53-4e9a-9a5b-542be95f5725" containerName="main"
Sep 19 13:06:34.742739 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742706    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9207e8cb-1c34-4477-bc65-5d60f024973d" containerName="main"
Sep 19 13:06:34.742739 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742711    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="9207e8cb-1c34-4477-bc65-5d60f024973d" containerName="main"
Sep 19 13:06:34.742836 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742782    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="9207e8cb-1c34-4477-bc65-5d60f024973d" containerName="main"
Sep 19 13:06:34.742836 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.742792    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7ed595f-9a53-4e9a-9a5b-542be95f5725" containerName="main"
Sep 19 13:06:34.747016 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.746999    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.749516 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.749495    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 19 13:06:34.750705 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.750688    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 19 13:06:34.750970 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.750947    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 19 13:06:34.751084 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.750966    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sr888\""
Sep 19 13:06:34.751257 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.751236    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 19 13:06:34.757873 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.757848    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"]
Sep 19 13:06:34.895279 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.895190    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-maas-api-tls\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.895279 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.895239    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-metrics-certs\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.895505 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.895330    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kngnq\" (UniqueName: \"kubernetes.io/projected/e009bb42-624f-4e46-a3de-0518352671de-kube-api-access-kngnq\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.895505 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.895361    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e009bb42-624f-4e46-a3de-0518352671de-openshift-service-ca\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.996038 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.996001    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e009bb42-624f-4e46-a3de-0518352671de-openshift-service-ca\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.996207 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.996068    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-maas-api-tls\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.996207 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.996116    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-metrics-certs\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.996207 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.996182    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kngnq\" (UniqueName: \"kubernetes.io/projected/e009bb42-624f-4e46-a3de-0518352671de-kube-api-access-kngnq\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.996802 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.996779    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e009bb42-624f-4e46-a3de-0518352671de-openshift-service-ca\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.998584 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.998563    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-metrics-certs\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:34.998694 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:34.998677    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-maas-api-tls\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:35.008639 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:35.008615    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kngnq\" (UniqueName: \"kubernetes.io/projected/e009bb42-624f-4e46-a3de-0518352671de-kube-api-access-kngnq\") pod \"maas-api-5598d8c9fb-s5wq7\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") " pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:35.057581 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:35.057546    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:35.328272 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:35.328242    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"]
Sep 19 13:06:35.330024 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:06:35.329999    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode009bb42_624f_4e46_a3de_0518352671de.slice/crio-c442d7653ca16dadbd7c9f7db6e2c8098541fe52bda7b4722e5750e51020531f WatchSource:0}: Error finding container c442d7653ca16dadbd7c9f7db6e2c8098541fe52bda7b4722e5750e51020531f: Status 404 returned error can't find the container with id c442d7653ca16dadbd7c9f7db6e2c8098541fe52bda7b4722e5750e51020531f
Sep 19 13:06:35.331412 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:35.331396    2571 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 19 13:06:35.389087 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:35.389054    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" event={"ID":"e009bb42-624f-4e46-a3de-0518352671de","Type":"ContainerStarted","Data":"c442d7653ca16dadbd7c9f7db6e2c8098541fe52bda7b4722e5750e51020531f"}
Sep 19 13:06:36.396236 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:36.396202    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" event={"ID":"e009bb42-624f-4e46-a3de-0518352671de","Type":"ContainerStarted","Data":"e7b3188473bcb9b5d9757ba887d2947aff59ee66110bce3966757e0fc7f67579"}
Sep 19 13:06:36.396638 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:36.396270    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:36.425576 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:36.425521    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" podStartSLOduration=1.983799001 podStartE2EDuration="2.42550236s" totalImagesPullingTime="441.703359ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-19 13:06:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 13:06:36.424497721 +0000 UTC m=+2479.784740661" watchObservedRunningTime="2026-09-19 13:06:36.42550236 +0000 UTC m=+2479.785745302"
Sep 19 13:06:42.028634 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:42.028599    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:06:42.406989 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:42.406911    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:49.228160 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:49.228123    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:06:52.356115 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:52.356080    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"]
Sep 19 13:06:52.356663 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:52.356291    2571 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" podUID="e009bb42-624f-4e46-a3de-0518352671de" containerName="maas-api" containerID="cri-o://e7b3188473bcb9b5d9757ba887d2947aff59ee66110bce3966757e0fc7f67579" gracePeriod=30
Sep 19 13:06:52.402159 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:52.402125    2571 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" podUID="e009bb42-624f-4e46-a3de-0518352671de" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.84:8443/health\": dial tcp 10.134.0.84:8443: connect: connection refused"
Sep 19 13:06:53.475232 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.475204    2571 generic.go:356] "Generic (PLEG): container finished" podID="e009bb42-624f-4e46-a3de-0518352671de" containerID="e7b3188473bcb9b5d9757ba887d2947aff59ee66110bce3966757e0fc7f67579" exitCode=0
Sep 19 13:06:53.475646 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.475272    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" event={"ID":"e009bb42-624f-4e46-a3de-0518352671de","Type":"ContainerDied","Data":"e7b3188473bcb9b5d9757ba887d2947aff59ee66110bce3966757e0fc7f67579"}
Sep 19 13:06:53.595701 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.595670    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:53.649434 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.649401    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-metrics-certs\") pod \"e009bb42-624f-4e46-a3de-0518352671de\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") "
Sep 19 13:06:53.649648 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.649448    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-maas-api-tls\") pod \"e009bb42-624f-4e46-a3de-0518352671de\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") "
Sep 19 13:06:53.649648 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.649500    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kngnq\" (UniqueName: \"kubernetes.io/projected/e009bb42-624f-4e46-a3de-0518352671de-kube-api-access-kngnq\") pod \"e009bb42-624f-4e46-a3de-0518352671de\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") "
Sep 19 13:06:53.649648 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.649543    2571 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e009bb42-624f-4e46-a3de-0518352671de-openshift-service-ca\") pod \"e009bb42-624f-4e46-a3de-0518352671de\" (UID: \"e009bb42-624f-4e46-a3de-0518352671de\") "
Sep 19 13:06:53.649969 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.649939    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e009bb42-624f-4e46-a3de-0518352671de-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e009bb42-624f-4e46-a3de-0518352671de" (UID: "e009bb42-624f-4e46-a3de-0518352671de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 19 13:06:53.651712 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.651678    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e009bb42-624f-4e46-a3de-0518352671de" (UID: "e009bb42-624f-4e46-a3de-0518352671de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 13:06:53.651807 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.651722    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e009bb42-624f-4e46-a3de-0518352671de" (UID: "e009bb42-624f-4e46-a3de-0518352671de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 19 13:06:53.651807 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.651733    2571 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e009bb42-624f-4e46-a3de-0518352671de-kube-api-access-kngnq" (OuterVolumeSpecName: "kube-api-access-kngnq") pod "e009bb42-624f-4e46-a3de-0518352671de" (UID: "e009bb42-624f-4e46-a3de-0518352671de"). InnerVolumeSpecName "kube-api-access-kngnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 19 13:06:53.750133 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.750063    2571 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-metrics-certs\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:53.750133 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.750086    2571 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e009bb42-624f-4e46-a3de-0518352671de-maas-api-tls\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:53.750133 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.750095    2571 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kngnq\" (UniqueName: \"kubernetes.io/projected/e009bb42-624f-4e46-a3de-0518352671de-kube-api-access-kngnq\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:53.750133 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:53.750106    2571 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e009bb42-624f-4e46-a3de-0518352671de-openshift-service-ca\") on node \"ip-10-0-131-60.ec2.internal\" DevicePath \"\""
Sep 19 13:06:54.480999 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:54.480970    2571 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"
Sep 19 13:06:54.481544 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:54.480964    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7" event={"ID":"e009bb42-624f-4e46-a3de-0518352671de","Type":"ContainerDied","Data":"c442d7653ca16dadbd7c9f7db6e2c8098541fe52bda7b4722e5750e51020531f"}
Sep 19 13:06:54.481544 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:54.481108    2571 scope.go:117] "RemoveContainer" containerID="e7b3188473bcb9b5d9757ba887d2947aff59ee66110bce3966757e0fc7f67579"
Sep 19 13:06:54.502866 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:54.502838    2571 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"]
Sep 19 13:06:54.506649 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:54.506627    2571 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5598d8c9fb-s5wq7"]
Sep 19 13:06:54.739486 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:54.739398    2571 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e009bb42-624f-4e46-a3de-0518352671de" path="/var/lib/kubelet/pods/e009bb42-624f-4e46-a3de-0518352671de/volumes"
Sep 19 13:06:57.232807 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:06:57.232777    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:07:02.629471 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:02.629375    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-gttp7"]
Sep 19 13:07:05.431576 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:05.431544    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98dc84ff9-s62vg_799bbd28-7132-4f43-acea-d8c6d48fd9f9/authorino/0.log"
Sep 19 13:07:11.620909 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:11.620868    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-qnwwc_5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d/manager/0.log"
Sep 19 13:07:11.628289 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:11.628263    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-qnwwc_5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d/copy-manifests/0.log"
Sep 19 13:07:12.287801 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:12.287768    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29830380-jlb4k_cd793ccf-3188-4a25-b45d-47210733472d/cleanup/0.log"
Sep 19 13:07:13.138212 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:13.138184    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98dc84ff9-s62vg_799bbd28-7132-4f43-acea-d8c6d48fd9f9/authorino/0.log"
Sep 19 13:07:13.475153 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:13.475124    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-2wtrl_cd9735f6-5b7a-47bc-bdc3-954c96ed7508/kuadrant-console-plugin/0.log"
Sep 19 13:07:13.716086 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:13.716057    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-gttp7_04627672-f1c7-4854-a09a-0b8217e717de/limitador/0.log"
Sep 19 13:07:14.544508 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:14.544448    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9bc57f57-wn4c7_18da225e-0f9e-4994-bf20-e2a4d9871678/kube-auth-proxy/0.log"
Sep 19 13:07:15.327397 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:15.327367    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7khrn_d78cb66f-3eca-485f-9587-7e774740a06a/main/0.log"
Sep 19 13:07:15.653371 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:15.653257    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hb6dk_634f3bc1-f890-492c-b2a5-fb76a2792522/main/0.log"
Sep 19 13:07:15.876354 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:15.876321    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-l6hl2_91852cea-879e-437d-bc83-d508c22e7806/main/0.log"
Sep 19 13:07:15.987071 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:15.987045    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-th7hj_9b506a77-e70c-4392-8d44-9fb534659db0/main/0.log"
Sep 19 13:07:19.128002 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:19.127968    2571 scope.go:117] "RemoveContainer" containerID="6f1eda21a4ee7f6d795cd1e85ee92682f39a1ab063bb2c2f8b1da18e539624ad"
Sep 19 13:07:19.135768 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:19.135747    2571 scope.go:117] "RemoveContainer" containerID="40cbf961a94758e152f070cab653f2bff90e09c7e413d29c69d9f0288d288b17"
Sep 19 13:07:21.799549 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:21.799515    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:07:21.898547 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:21.898509    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:07:25.306965 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:25.306930    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wf7d9_ae5d7b23-07f5-47d3-af44-d1cb64091bca/global-pull-secret-syncer/0.log"
Sep 19 13:07:25.368501 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:25.368476    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log"
Sep 19 13:07:25.501526 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:25.501484    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-60.ec2.internal_769ded29dc5f76d39a7c0031f5b9ed81/haproxy/0.log"
Sep 19 13:07:29.982785 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:29.982733    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-98dc84ff9-s62vg_799bbd28-7132-4f43-acea-d8c6d48fd9f9/authorino/0.log"
Sep 19 13:07:30.074557 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:30.074530    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-2wtrl_cd9735f6-5b7a-47bc-bdc3-954c96ed7508/kuadrant-console-plugin/0.log"
Sep 19 13:07:30.155552 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:30.155522    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-gttp7_04627672-f1c7-4854-a09a-0b8217e717de/limitador/0.log"
Sep 19 13:07:32.051486 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:32.051447    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7894ffc44b-5kv7h_b1f1ddff-9e56-4ae8-9ef2-8dcb2a86ce2b/metrics-server/0.log"
Sep 19 13:07:32.296691 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:32.296649    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbg5g_e243b504-1c74-46e3-9eae-f0901dfe5fce/node-exporter/0.log"
Sep 19 13:07:32.318869 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:32.318790    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbg5g_e243b504-1c74-46e3-9eae-f0901dfe5fce/kube-rbac-proxy/0.log"
Sep 19 13:07:32.344151 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:32.344127    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cbg5g_e243b504-1c74-46e3-9eae-f0901dfe5fce/init-textfile/0.log"
Sep 19 13:07:32.659395 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:32.659305    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-kq4jv_79147762-c8d6-4db2-a12e-64739ddfd95b/prometheus-operator/0.log"
Sep 19 13:07:32.676066 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:32.676042    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-kq4jv_79147762-c8d6-4db2-a12e-64739ddfd95b/kube-rbac-proxy/0.log"
Sep 19 13:07:35.390482 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.390429    2571 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"]
Sep 19 13:07:35.391075 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.390883    2571 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e009bb42-624f-4e46-a3de-0518352671de" containerName="maas-api"
Sep 19 13:07:35.391075 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.390903    2571 state_mem.go:107] "Deleted CPUSet assignment" podUID="e009bb42-624f-4e46-a3de-0518352671de" containerName="maas-api"
Sep 19 13:07:35.391075 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.390991    2571 memory_manager.go:368] "RemoveStaleState removing state" podUID="e009bb42-624f-4e46-a3de-0518352671de" containerName="maas-api"
Sep 19 13:07:35.394096 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.394076    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.396434 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.396414    2571 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rrtv4\"/\"default-dockercfg-xbfxk\""
Sep 19 13:07:35.396775 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.396761    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rrtv4\"/\"openshift-service-ca.crt\""
Sep 19 13:07:35.397661 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.397648    2571 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rrtv4\"/\"kube-root-ca.crt\""
Sep 19 13:07:35.401547 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.401509    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-proc\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.401672 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.401576    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-sys\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.401672 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.401636    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jn8m\" (UniqueName: \"kubernetes.io/projected/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-kube-api-access-6jn8m\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.401760 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.401691    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-podres\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.401760 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.401725    2571 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-lib-modules\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.404374 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.404353    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"]
Sep 19 13:07:35.502795 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502756    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jn8m\" (UniqueName: \"kubernetes.io/projected/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-kube-api-access-6jn8m\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.502795 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502802    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-podres\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502826    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-lib-modules\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502855    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-proc\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502877    2571 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-sys\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502957    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-proc\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502971    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-podres\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502986    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-lib-modules\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.503032 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.502964    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-sys\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.512177 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.512150    2571 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jn8m\" (UniqueName: \"kubernetes.io/projected/9fc95836-9f16-4f7c-9e25-b9f4270fffa7-kube-api-access-6jn8m\") pod \"perf-node-gather-daemonset-bz55m\" (UID: \"9fc95836-9f16-4f7c-9e25-b9f4270fffa7\") " pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.704161 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.704133    2571 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:35.974642 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:35.974427    2571 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"]
Sep 19 13:07:35.977360 ip-10-0-131-60 kubenswrapper[2571]: W0919 13:07:35.977332    2571 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9fc95836_9f16_4f7c_9e25_b9f4270fffa7.slice/crio-f1e16e75c16ebd2626f563694344d6478ee3c212ca369230857065369f6bda3c WatchSource:0}: Error finding container f1e16e75c16ebd2626f563694344d6478ee3c212ca369230857065369f6bda3c: Status 404 returned error can't find the container with id f1e16e75c16ebd2626f563694344d6478ee3c212ca369230857065369f6bda3c
Sep 19 13:07:36.671889 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:36.671851    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m" event={"ID":"9fc95836-9f16-4f7c-9e25-b9f4270fffa7","Type":"ContainerStarted","Data":"08ef8bdc0f3c7699a68b5b2eb6b1d6d44357d608d6f338e323a41a6b7202e108"}
Sep 19 13:07:36.672350 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:36.671908    2571 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m" event={"ID":"9fc95836-9f16-4f7c-9e25-b9f4270fffa7","Type":"ContainerStarted","Data":"f1e16e75c16ebd2626f563694344d6478ee3c212ca369230857065369f6bda3c"}
Sep 19 13:07:36.672350 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:36.672012    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:36.688650 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:36.688604    2571 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m" podStartSLOduration=1.68858888 podStartE2EDuration="1.68858888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-19 13:07:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-19 13:07:36.6878308 +0000 UTC m=+2540.048073743" watchObservedRunningTime="2026-09-19 13:07:36.68858888 +0000 UTC m=+2540.048831821"
Sep 19 13:07:37.474382 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:37.474353    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c4bvb_b2e90124-26b9-40ad-9281-d14a5bcc74e4/dns/0.log"
Sep 19 13:07:37.496653 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:37.496616    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-c4bvb_b2e90124-26b9-40ad-9281-d14a5bcc74e4/kube-rbac-proxy/0.log"
Sep 19 13:07:37.681861 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:37.681835    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-qx4xs_80304af3-45a3-466b-a4ea-d7ac748811f3/dns-node-resolver/0.log"
Sep 19 13:07:38.263855 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:38.263826    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jpl8l_e2291616-999a-4193-8882-f43d2174c60e/node-ca/0.log"
Sep 19 13:07:40.081000 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:40.080969    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9bc57f57-wn4c7_18da225e-0f9e-4994-bf20-e2a4d9871678/kube-auth-proxy/0.log"
Sep 19 13:07:41.152827 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:41.152801    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s882b_ccd17cda-e0b9-44c7-9375-7769b1a38a39/serve-healthcheck-canary/0.log"
Sep 19 13:07:41.764066 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:41.764037    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qnpdg_29a6e79e-a793-4ed7-825c-284e062e97fc/kube-rbac-proxy/0.log"
Sep 19 13:07:41.786717 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:41.786694    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qnpdg_29a6e79e-a793-4ed7-825c-284e062e97fc/exporter/0.log"
Sep 19 13:07:41.806733 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:41.806701    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qnpdg_29a6e79e-a793-4ed7-825c-284e062e97fc/extractor/0.log"
Sep 19 13:07:42.685823 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:42.685799    2571 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rrtv4/perf-node-gather-daemonset-bz55m"
Sep 19 13:07:42.713182 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:42.713147    2571 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-jr2dg_0e407c77-d390-4e67-ae03-f083fa4acf52/konnectivity-agent/0.log" limit=1024
Sep 19 13:07:44.892308 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:44.892275    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-qnwwc_5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d/manager/0.log"
Sep 19 13:07:44.908926 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:44.908904    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-774bc4964b-qnwwc_5bf1c9fe-a4bd-4cb1-9af9-e2c735d3032d/copy-manifests/0.log"
Sep 19 13:07:46.263500 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:46.263455    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6bdf74d45f-hw9cv_c61c76c5-af38-451f-a0d6-43de4887bb64/manager/0.log"
Sep 19 13:07:46.637279 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:46.637252    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9bc57f57-wn4c7_18da225e-0f9e-4994-bf20-e2a4d9871678/kube-auth-proxy/0.log"
Sep 19 13:07:49.140386 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:49.140350    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5c9bc57f57-wn4c7_18da225e-0f9e-4994-bf20-e2a4d9871678/kube-auth-proxy/0.log"
Sep 19 13:07:51.483281 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:51.483193    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-tkwbm_ee553785-32df-49c6-b381-cfe61b9b9153/migrator/0.log"
Sep 19 13:07:51.547006 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:51.546976    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-tkwbm_ee553785-32df-49c6-b381-cfe61b9b9153/graceful-termination/0.log"
Sep 19 13:07:52.908666 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:52.908628    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/kube-multus-additional-cni-plugins/0.log"
Sep 19 13:07:52.939798 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:52.939773    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/egress-router-binary-copy/0.log"
Sep 19 13:07:52.995874 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:52.995848    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/cni-plugins/0.log"
Sep 19 13:07:53.057257 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:53.057231    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/bond-cni-plugin/0.log"
Sep 19 13:07:53.098258 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:53.098232    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/routeoverride-cni/0.log"
Sep 19 13:07:53.153957 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:53.153933    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/whereabouts-cni-bincopy/0.log"
Sep 19 13:07:53.204506 ip-10-0-131-60 kubenswrapper[2571]: I0919 13:07:53.204483    2571 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7qjb8_638fdc50-3117-472b-a7f7-133a1c8d015a/whereabouts-cni/0.log"