Sep 11 17:36:50.401262 ip-10-0-128-237 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 17:36:50.401273 ip-10-0-128-237 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 17:36:50.401280 ip-10-0-128-237 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 17:36:50.401516 ip-10-0-128-237 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 17:37:00.414547 ip-10-0-128-237 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 17:37:00.414565 ip-10-0-128-237 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ce5c467adff943f1bf1edf33b8945480 --
Sep 11 17:38:37.030628 ip-10-0-128-237 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 17:38:37.439831 ip-10-0-128-237 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:38:37.439831 ip-10-0-128-237 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 17:38:37.439831 ip-10-0-128-237 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:38:37.439831 ip-10-0-128-237 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 17:38:37.439831 ip-10-0-128-237 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 17:38:37.440879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.440792    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 17:38:37.444102 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444087    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:37.444102 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444103    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444107    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444110    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444114    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444118    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444122    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444125    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444128    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444131    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444134    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444137    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444140    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444143    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444146    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444149    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444152    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444156    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444167    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444170    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:37.444166 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444174    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444177    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444180    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444182    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444185    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444187    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444190    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444193    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444195    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444198    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444200    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444203    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444206    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444208    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444211    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444214    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444216    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444219    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444221    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444224    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:37.444672 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444226    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444229    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444231    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444234    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444236    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444239    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444241    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444244    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444246    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444249    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444251    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444254    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444257    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444261    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444264    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444267    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444270    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444272    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444275    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444278    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:37.445447 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444281    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444284    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444286    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444290    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444292    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444295    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444299    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444302    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444304    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444307    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444310    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444313    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444315    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444318    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444320    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444323    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444326    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444328    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444331    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444334    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:37.445941 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444336    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444358    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444363    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444367    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444371    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444377    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444380    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444384    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444386    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444389    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444392    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444394    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444397    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444399    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444402    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444405    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444407    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444410    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:37.446445 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.444414    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446467    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446479    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446483    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446486    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446489    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446492    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446495    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446497    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446500    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446503    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446505    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446508    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446510    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446513    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446515    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446518    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446521    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446524    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446526    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:37.446878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446529    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446533    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446537    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446541    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446544    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446547    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446549    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446552    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446555    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446557    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446560    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446562    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446565    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446572    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446575    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446578    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446580    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446583    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446586    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446588    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:37.447421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446591    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446595    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446599    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446602    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446605    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446608    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446610    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446613    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446616    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446618    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446621    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446624    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446626    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446630    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446634    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446637    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446640    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446642    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446645    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:37.447995 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446648    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446650    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446653    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446655    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446658    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446661    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446665    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446668    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446670    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446673    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446676    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446678    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446681    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446683    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446686    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446689    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446691    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446694    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446696    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446699    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:37.448495 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446701    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446704    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446707    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446709    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446711    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446721    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446723    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446726    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446729    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446732    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446734    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446737    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446739    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446742    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446744    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446747    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446750    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446752    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446755    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446758    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:37.448999 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.446761    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447172    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447176    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447179    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447182    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447184    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447187    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447190    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447193    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447195    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447198    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447200    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447203    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447206    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447208    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447210    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447213    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447224    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447227    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447229    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:37.449499 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447232    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447235    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447238    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447240    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447243    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447245    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447248    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447251    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447255    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447259    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447262    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447266    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447268    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447271    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447274    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447277    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447280    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447283    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447285    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:37.450000 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447288    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447291    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447294    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447296    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447299    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447302    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447304    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447307    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447310    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447314    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447317    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447326    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447329    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447332    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447336    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447338    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447341    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447361    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447365    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:37.450475 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447367    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447370    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447372    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447375    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447378    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447382    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447385    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447388    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447390    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447393    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447395    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447398    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447401    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447403    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447406    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447409    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447413    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447416    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447419    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447421    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:37.450940 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447424    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447427    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447430    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447433    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447435    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447444    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447447    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447450    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447452    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447455    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447457    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447460    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447463    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447466    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447468    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447471    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447474    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447477    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447479    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447482    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:37.451440 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447484    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.447487    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447563    2582 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447608    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447614    2582 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447619    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447623    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447626    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447630    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447634    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447638    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447641    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447644    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447647    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447650    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447654    2582 flags.go:64] FLAG: --cgroup-root=""
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447657    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447660    2582 flags.go:64] FLAG: --client-ca-file=""
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447669    2582 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447672    2582 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447680    2582 flags.go:64] FLAG: --cluster-domain=""
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447682    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447685    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447689    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 17:38:37.451967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447692    2582 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447696    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447699    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447702    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447705    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447708    2582 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447711    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447714    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447717    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447719    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447723    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447727    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447730    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447733    2582 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447736    2582 flags.go:64] FLAG: --enable-server="true"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447739    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447745    2582 flags.go:64] FLAG: --event-burst="100"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447748    2582 flags.go:64] FLAG: --event-qps="50"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447751    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447754    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447756    2582 flags.go:64] FLAG: --eviction-hard=""
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447760    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447763    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447766    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447769    2582 flags.go:64] FLAG: --eviction-soft=""
Sep 11 17:38:37.452590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447771    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447774    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447782    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447785    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447788    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447791    2582 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447796    2582 flags.go:64] FLAG: --feature-gates=""
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447800    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447803    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447806    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447809    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447812    2582 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447815    2582 flags.go:64] FLAG: --help="false"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447818    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447821    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447824    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447827    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447830    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447834    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447837    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447839    2582 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447842    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447845    2582 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 17:38:37.453198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447848    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447851    2582 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447854    2582 flags.go:64] FLAG: --kube-reserved=""
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447857    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447860    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447863    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447866    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447869    2582 flags.go:64] FLAG: --lock-file=""
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447871    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447874    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447877    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447882    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447891    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447894    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447897    2582 flags.go:64] FLAG: --logging-format="text"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447901    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447905    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447908    2582 flags.go:64] FLAG: --manifest-url=""
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447910    2582 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447915    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447918    2582 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447921    2582 flags.go:64] FLAG: --max-pods="110"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447924    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447927    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447930    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 17:38:37.453816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447933    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447938    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447941    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447944    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447951    2582 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447954    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447957    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447960    2582 flags.go:64] FLAG: --pod-cidr=""
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447963    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447968    2582 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447971    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447974    2582 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447977    2582 flags.go:64] FLAG: --port="10250"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447980    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447983    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aa5afecf50184ca1"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447986    2582 flags.go:64] FLAG: --qos-reserved=""
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447989    2582 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447992    2582 flags.go:64] FLAG: --register-node="true"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447995    2582 flags.go:64] FLAG: --register-with-taints=""
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.447998    2582 flags.go:64] FLAG: --registry-burst="10"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448007    2582 flags.go:64] FLAG: --registry-qps="5"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448010    2582 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448013    2582 flags.go:64] FLAG: --reserved-memory=""
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448018    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448021    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 17:38:37.454436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448024    2582 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448027    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448030    2582 flags.go:64] FLAG: --runonce="false"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448033    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448036    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448039    2582 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448042    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448044    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448049    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448052    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448055    2582 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448058    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448060    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448063    2582 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448066    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448069    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448072    2582 flags.go:64] FLAG: --system-cgroups=""
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448074    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448080    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448083    2582 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448085    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448092    2582 flags.go:64] FLAG: --tls-min-version=""
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448094    2582 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448097    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448100    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 17:38:37.455038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448103    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448106    2582 flags.go:64] FLAG: --v="2"
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448110    2582 flags.go:64] FLAG: --version="false"
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448120    2582 flags.go:64] FLAG: --vmodule=""
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448124    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.448129    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448265    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448268    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448271    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448274    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448277    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448279    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448282    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448285    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448287    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448291    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448294    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448297    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448300    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448302    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448305    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:37.455697 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448308    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448310    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448313    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448315    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448318    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448321    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448323    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448326    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448329    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448332    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448334    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448337    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448364    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448368    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448370    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448380    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448384    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448387    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448390    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448392    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:37.456203 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448395    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448397    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448400    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448402    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448405    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448409    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448414    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448417    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448420    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448423    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448425    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448429    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448431    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448434    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448437    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448440    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448444    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448447    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448450    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:37.456711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448453    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448456    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448459    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448461    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448464    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448467    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448470    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448472    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448475    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448485    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448488    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448492    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448495    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448498    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448501    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448504    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448507    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448509    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448512    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:37.457186 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448516    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448519    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448522    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448524    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448527    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448530    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448532    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448535    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448537    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448540    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448542    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448545    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448548    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448551    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448553    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448556    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448558    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448561    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448563    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448566    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:37.457698 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448568    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448571    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448574    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448584    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448586    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.448589    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.449322    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.457204    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.457220    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.457237    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.457243    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457297    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457301    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457305    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:37.458194 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457308    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457311    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457315    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457320    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457323    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457326    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457329    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457333    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457336    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457339    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457342    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457360    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457365    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457368    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457371    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457374    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457376    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457379    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457382    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:37.458558 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457384    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457387    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457390    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457392    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457395    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457398    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457401    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457405    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457408    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457411    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457414    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457417    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457419    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457422    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457425    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457427    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457430    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457434    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457438    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457441    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:37.459047 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457443    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457446    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457449    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457451    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457454    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457457    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457459    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457462    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457464    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457467    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457469    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457473    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457476    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457480    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457483    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457486    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457489    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457492    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457494    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:37.459564 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457497    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457501    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457504    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457506    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457509    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457512    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457515    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457517    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457520    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457522    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457525    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457528    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457531    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457533    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457536    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457538    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457541    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457543    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457546    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:37.460026 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457549    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457552    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457554    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457557    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457560    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457562    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457565    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457568    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457570    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457573    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457575    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457578    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457580    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457583    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457586    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457593    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457596    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457599    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:37.460513 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457601    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.457607    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457723    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457728    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457731    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457734    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457737    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457740    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457742    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457745    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457747    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457750    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457752    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457755    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457758    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 17:38:37.460961 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457761    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457763    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457766    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457768    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457771    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457773    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457776    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457779    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457781    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457784    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457786    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457789    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457791    2582 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457795    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457797    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457801    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457803    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457806    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457809    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457811    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 17:38:37.461338 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457814    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457817    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457819    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457822    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457824    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457828    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457832    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457835    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457838    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457841    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457844    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457847    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457850    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457853    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457856    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457859    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457862    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457866    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457869    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 17:38:37.461953 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457872    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457875    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457878    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457881    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457884    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457886    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457889    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457892    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457895    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457898    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457901    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457904    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457907    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457910    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457913    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457916    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457919    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457922    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457925    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 17:38:37.462421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457927    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457930    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457933    2582 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457936    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457939    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457942    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457945    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457948    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457951    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457954    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457956    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457959    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457962    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457964    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457968    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457971    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457973    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457976    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457978    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457981    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 17:38:37.462893 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457983    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457986    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457989    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457992    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457996    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.457999    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.458002    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:37.458005    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.458010    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.458147    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.460591    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.461716    2582 server.go:1019] "Starting client certificate rotation"
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.461822    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 17:38:37.463388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.461857    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 17:38:37.464855 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.464834    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 17:38:37.472592 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.469067    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 17:38:37.482121 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.482102    2582 log.go:25] "Validated CRI v1 runtime API"
Sep 11 17:38:37.485368 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.485339    2582 log.go:25] "Validated CRI v1 image API"
Sep 11 17:38:37.486191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.486177    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 17:38:37.489907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.489886    2582 fs.go:135] Filesystem UUIDs: map[30908efb-db74-4f16-bae7-208825a0120e:/dev/nvme0n1p3 3f782852-5373-444d-90c8-a955f44cb35a:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Sep 11 17:38:37.489968 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.489905    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 17:38:37.491020 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.491001    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 17:38:37.496013 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.495908    2582 manager.go:216] Machine: {Timestamp:2026-09-11 17:38:37.494537764 +0000 UTC m=+0.364580700 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3128513 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21c38511a4b5b276fb91cbbe8b7f29 SystemUUID:ec21c385-11a4-b5b2-76fb-91cbbe8b7f29 BootID:ce5c467a-dff9-43f1-bf1e-df33b8945480 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4d:a7:66:eb:9f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4d:a7:66:eb:9f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:62:0a:59:74:40:6b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 17:38:37.496013 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.496013    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 17:38:37.496138 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.496127    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 17:38:37.499556 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.499535    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 17:38:37.499695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.499560    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-237.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 17:38:37.501114 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.501104    2582 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 17:38:37.501149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.501116    2582 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 17:38:37.501149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.501124    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 17:38:37.501936 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.501926    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 17:38:37.503194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.503184    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 17:38:37.508729 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508717    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 17:38:37.508844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508835    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 17:38:37.508896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508888    2582 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 17:38:37.508928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508900    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 17:38:37.508928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508915    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 17:38:37.508928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508925    2582 kubelet.go:398] "Adding apiserver pod source"
Sep 11 17:38:37.509024 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.508934    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 17:38:37.509420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.509408    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 17:38:37.509478 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.509428    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 17:38:37.511886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.511856    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 17:38:37.514702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.514688    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 17:38:37.514754 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.514717    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 17:38:37.516746 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516735    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 17:38:37.516782 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516752    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 17:38:37.516782 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516758    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 17:38:37.516782 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516764    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 17:38:37.516782 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516772    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 17:38:37.516782 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516779    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 17:38:37.516923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516792    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 17:38:37.516923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516798    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 17:38:37.516923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516805    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 17:38:37.516923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516811    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 17:38:37.516923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516827    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 17:38:37.516923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.516833    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 17:38:37.517762 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.517751    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 17:38:37.517762 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.517762    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 17:38:37.519288 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.519258    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 17:38:37.519472 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.519446    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 17:38:37.519545 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.519527    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-237.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 17:38:37.522180 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.522162    2582 server.go:1268] "Started kubelet"
Sep 11 17:38:37.522280 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.522220    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 17:38:37.522339 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.522269    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 17:38:37.522339 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.522323    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 17:38:37.522940 ip-10-0-128-237 systemd[1]: Started Kubernetes Kubelet.
Sep 11 17:38:37.523994 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.523850    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 17:38:37.525107 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.525094    2582 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 17:38:37.529442 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.529421    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 17:38:37.530922 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.530908    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 17:38:37.530990 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.530920    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 17:38:37.531038 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.530970    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:37.531697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531673    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 17:38:37.531780 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531675    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 17:38:37.531780 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531745    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 17:38:37.531889 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531852    2582 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 17:38:37.531889 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531865    2582 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 17:38:37.531889 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531853    2582 factory.go:55] Registering systemd factory
Sep 11 17:38:37.531889 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.531886    2582 factory.go:223] Registration of the systemd container factory successfully
Sep 11 17:38:37.532115 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.532097    2582 factory.go:153] Registering CRI-O factory
Sep 11 17:38:37.532115 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.532117    2582 factory.go:223] Registration of the crio container factory successfully
Sep 11 17:38:37.532234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.532168    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 17:38:37.532234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.532188    2582 factory.go:103] Registering Raw factory
Sep 11 17:38:37.532234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.532203    2582 manager.go:1196] Started watching for new ooms in manager
Sep 11 17:38:37.532682 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.532669    2582 manager.go:349] Starting recovery of all containers
Sep 11 17:38:37.538099 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.537826    2582 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-237.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 17:38:37.538099 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.537913    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 17:38:37.539416 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.538019    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5c64e3ce  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.522125774 +0000 UTC m=+0.392168710,LastTimestamp:2026-09-11 17:38:37.522125774 +0000 UTC m=+0.392168710,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.543920 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.543902    2582 manager.go:354] Recovery completed
Sep 11 17:38:37.550699 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.550685    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.552153 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552136    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.552231 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552168    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.552231 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552183    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.552861 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552844    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 17:38:37.552861 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552856    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.553010 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552857    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 17:38:37.553010 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.552920    2582 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 17:38:37.553602 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.553586    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.553687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.553611    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.553687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.553621    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.554490 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.554472    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q2wrh"
Sep 11 17:38:37.554685 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.554599    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5e2f11ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.552153087 +0000 UTC m=+0.422196033,LastTimestamp:2026-09-11 17:38:37.552153087 +0000 UTC m=+0.422196033,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.562716 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.562642    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5e2f68a0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.552175264 +0000 UTC m=+0.422218202,LastTimestamp:2026-09-11 17:38:37.552175264 +0000 UTC m=+0.422218202,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.563225 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.563212    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-q2wrh"
Sep 11 17:38:37.567263 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.567192    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5e2f9dbc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.55218886 +0000 UTC m=+0.422231801,LastTimestamp:2026-09-11 17:38:37.55218886 +0000 UTC m=+0.422231801,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.573681 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.573607    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18d454ea5e2f11ff\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5e2f11ff  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.552153087 +0000 UTC m=+0.422196033,LastTimestamp:2026-09-11 17:38:37.553599361 +0000 UTC m=+0.423642297,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.584073 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.583992    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18d454ea5e2f68a0\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5e2f68a0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.552175264 +0000 UTC m=+0.422218202,LastTimestamp:2026-09-11 17:38:37.553615515 +0000 UTC m=+0.423658451,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.610106 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.593179    2582 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-237.ec2.internal.18d454ea5e2f9dbc\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-237.ec2.internal.18d454ea5e2f9dbc  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-237.ec2.internal,UID:ip-10-0-128-237.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-237.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-237.ec2.internal,},FirstTimestamp:2026-09-11 17:38:37.55218886 +0000 UTC m=+0.422231801,LastTimestamp:2026-09-11 17:38:37.553624512 +0000 UTC m=+0.423667448,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-237.ec2.internal,}"
Sep 11 17:38:37.631320 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.631298    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:37.698955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.698904    2582 policy_none.go:49] "None policy: Start"
Sep 11 17:38:37.698955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.698922    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 17:38:37.698955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.698934    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 17:38:37.701126 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.701111    2582 policy_none.go:47] "Start"
Sep 11 17:38:37.732156 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.732139    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:37.738658 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.738630    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 17:38:37.739787 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.739765    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 17:38:37.739860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.739812    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 17:38:37.739860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.739832    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 17:38:37.739949 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.739865    2582 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Sep 11 17:38:37.741801 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.741786    2582 manager.go:337] "Starting Device Plugin manager"
Sep 11 17:38:37.741882 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.741822    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 17:38:37.741882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.741832    2582 server.go:87] "Starting device plugin registration server"
Sep 11 17:38:37.742103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.742089    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 17:38:37.742160 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.742108    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 17:38:37.742468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.742229    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 17:38:37.742468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.742331    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 17:38:37.742468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.742340    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 17:38:37.742985 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.742947    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 17:38:37.743085 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.743006    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:37.747531 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.747516    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:37.749795 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.749780    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.840497 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.840451    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"]
Sep 11 17:38:37.840637 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.840562    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.841654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.841634    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.841720 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.841672    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.841720 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.841684    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.842748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.842734    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.843453 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.843440    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.843509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.843467    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.843509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.843480    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.843509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.843499    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.844105 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844093    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.844226 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844213    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.844313 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844243    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.844819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844802    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.844819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844824    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.844924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844840    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.844924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844826    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.844924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844875    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.844924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.844884    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.847108 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.847092    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.847186 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.847126    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:37.847770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.847756    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:37.847858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.847784    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:37.847858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.847797    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:37.853569 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.853557    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.853633 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.853575    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-237.ec2.internal\": node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:37.863887 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.863871    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.867397 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.867382    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-237.ec2.internal\" not found" node="ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.881466 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.881449    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:37.933779 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.933756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a2ad4dde9645228f8d73db0210fa56c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"6a2ad4dde9645228f8d73db0210fa56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.933860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.933784    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a2ad4dde9645228f8d73db0210fa56c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"6a2ad4dde9645228f8d73db0210fa56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.933860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:37.933800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c6603c74816e2e6b08fd8fe02fdf018-config\") pod \"kube-apiserver-proxy-ip-10-0-128-237.ec2.internal\" (UID: \"8c6603c74816e2e6b08fd8fe02fdf018\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:37.982258 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:37.982240    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.034794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.034769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a2ad4dde9645228f8d73db0210fa56c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"6a2ad4dde9645228f8d73db0210fa56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.034860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.034797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a2ad4dde9645228f8d73db0210fa56c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"6a2ad4dde9645228f8d73db0210fa56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.034860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.034816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c6603c74816e2e6b08fd8fe02fdf018-config\") pod \"kube-apiserver-proxy-ip-10-0-128-237.ec2.internal\" (UID: \"8c6603c74816e2e6b08fd8fe02fdf018\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.034860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.034854    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c6603c74816e2e6b08fd8fe02fdf018-config\") pod \"kube-apiserver-proxy-ip-10-0-128-237.ec2.internal\" (UID: \"8c6603c74816e2e6b08fd8fe02fdf018\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.034951 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.034872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/6a2ad4dde9645228f8d73db0210fa56c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"6a2ad4dde9645228f8d73db0210fa56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.034951 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.034880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6a2ad4dde9645228f8d73db0210fa56c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal\" (UID: \"6a2ad4dde9645228f8d73db0210fa56c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.082900 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.082872    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.166333 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.166303    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.169624 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.169598    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.183290 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.183163    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.283853 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.283826    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.384150 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.384111    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.462026 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.461987    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 17:38:38.462524 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.462167    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 17:38:38.484458 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.484420    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.585652 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.585561    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.652583 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.652552    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 17:38:38.653949 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.653926    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 17:38:38.654054 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.653967    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 17:38:38.654054 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.653982    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeHasSufficientPID"
Sep 11 17:38:38.686174 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:38.686138    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-237.ec2.internal\" not found"
Sep 11 17:38:38.703509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.703470    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 17:33:37 +0000 UTC" deadline="2028-06-07 09:44:58.641064208 +0000 UTC"
Sep 11 17:38:38.703509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.703505    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15232h6m19.937562289s"
Sep 11 17:38:38.716304 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.716281    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:38.725194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.725175    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:38.731415 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.731398    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.747541 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.747522    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 17:38:38.750540 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.750516    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal"
Sep 11 17:38:38.761702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.761674    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 17:38:38.789058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.789032    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:38.805853 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:38.805822    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c6603c74816e2e6b08fd8fe02fdf018.slice/crio-d585f6b537cc8f5c3b8158a2a7c45321a68955e61b24209c824c5f67e80ad114 WatchSource:0}: Error finding container d585f6b537cc8f5c3b8158a2a7c45321a68955e61b24209c824c5f67e80ad114: Status 404 returned error can't find the container with id d585f6b537cc8f5c3b8158a2a7c45321a68955e61b24209c824c5f67e80ad114
Sep 11 17:38:38.806227 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:38.806211    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a2ad4dde9645228f8d73db0210fa56c.slice/crio-da3134995bd6a89e2a080d012dcd99860e60eaec1fedb77c3676768cc993d6e4 WatchSource:0}: Error finding container da3134995bd6a89e2a080d012dcd99860e60eaec1fedb77c3676768cc993d6e4: Status 404 returned error can't find the container with id da3134995bd6a89e2a080d012dcd99860e60eaec1fedb77c3676768cc993d6e4
Sep 11 17:38:38.811131 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:38.811118    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:38:39.510001 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.509971    2582 apiserver.go:52] "Watching apiserver"
Sep 11 17:38:39.516529 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.516500    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 17:38:39.517011 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.516978    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-7h22x","openshift-network-operator/iptables-alerter-9x8c4","kube-system/global-pull-secret-syncer-cgzbn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9","openshift-dns/node-resolver-vv7sb","openshift-multus/multus-5ct78","openshift-network-diagnostics/network-check-target-rr858","openshift-ovn-kubernetes/ovnkube-node-nwqjv","kube-system/konnectivity-agent-7c6kr","kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal","openshift-cluster-node-tuning-operator/tuned-gvj4q","openshift-image-registry/node-ca-xx9jz","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal","openshift-multus/multus-additional-cni-plugins-pmgpd"]
Sep 11 17:38:39.519858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.519801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:39.520015 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.519908    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:39.522257 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.522228    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.526037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.524766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.526037 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.524841    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:39.526037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.524903    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.526037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.524964    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8nnz8\""
Sep 11 17:38:39.526037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.524903    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.526037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.525183    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 17:38:39.527077 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.527057    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.529561 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.529350    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-48mhc\""
Sep 11 17:38:39.529667 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.529615    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.530041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.529782    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.530041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.529861    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.530041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.529884    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 17:38:39.531685 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.531628    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 17:38:39.531860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.531742    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vkpkh\""
Sep 11 17:38:39.532004 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.531987    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.532112 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.532012    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.532319 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.532236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.534461 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.534410    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:39.534589 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.534476    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:39.534655 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.534598    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 17:38:39.534824 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.534804    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 17:38:39.534902 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.534831    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.534985 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.534967    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-tbbxd\""
Sep 11 17:38:39.535042 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.534987    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.537049 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.537023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.539562 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.539119    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.539562 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.539253    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 17:38:39.539854 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.539748    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.540215 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.540198    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 17:38:39.540491 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.540473    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.540627 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.540601    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 17:38:39.540730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.540671    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 17:38:39.540730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.540526    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xqr5d\""
Sep 11 17:38:39.542223 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.542203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.543999    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jk9gf\""
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.544202    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.544552    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.544856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgc5x\" (UniqueName: \"kubernetes.io/projected/197940f3-4a76-4c8f-bf3f-c4adcf446604-kube-api-access-xgc5x\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.544873    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tf97q\""
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.544905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-hosts-file\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.544948    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89278d92-d875-487f-8fdb-6256db682130-host-slash\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545041    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-system-cni-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-conf-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545108    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx2hz\" (UniqueName: \"kubernetes.io/projected/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-kube-api-access-jx2hz\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545137    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545155    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545149    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/89278d92-d875-487f-8fdb-6256db682130-iptables-alerter-script\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/59f9e5b9-528f-4400-a787-3c339153c3b4-kubelet-config\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545449    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-device-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545538    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tbqp\" (UniqueName: \"kubernetes.io/projected/c6b182ac-212d-4b50-8cfc-36909e6f8d44-kube-api-access-8tbqp\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.545587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-cni-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545683    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545781    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-cnibin\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545815    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcct4\" (UniqueName: \"kubernetes.io/projected/89278d92-d875-487f-8fdb-6256db682130-kube-api-access-pcct4\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.545847    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-etc-selinux\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546020    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-sys-fs\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39334c02-daa5-439d-9780-386add54e04b-cni-binary-copy\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546139    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-etc-kubernetes\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546175    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546187    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/59f9e5b9-528f-4400-a787-3c339153c3b4-dbus\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546228    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-netns\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-cni-bin\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-multus-certs\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.546479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-tmp-dir\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.547129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546543    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-registration-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.547129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-cni-multus\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.546910    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-kubelet\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547019    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/39334c02-daa5-439d-9780-386add54e04b-multus-daemon-config\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:39.547369 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547192    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.547369 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547266    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-socket-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.547493 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547348    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-k8s-cni-cncf-io\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547493 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547432    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-hostroot\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547493 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547467    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-socket-dir-parent\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlzlw\" (UniqueName: \"kubernetes.io/projected/39334c02-daa5-439d-9780-386add54e04b-kube-api-access-dlzlw\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.547649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.547624    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 17:38:39.548551 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.548530    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 17:38:39.549391 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.549256    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 17:38:39.549391 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.549402    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 17:38:39.549565 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.549430    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lnz8n\""
Sep 11 17:38:39.549765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.549745    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.551698 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.551663    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-glnbx\""
Sep 11 17:38:39.551802 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.551714    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 17:38:39.551865 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.551667    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 17:38:39.574787 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.574754    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hv5rw"
Sep 11 17:38:39.584338 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.584117    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 17:38:39.584338 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.584304    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hv5rw"
Sep 11 17:38:39.633133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.633091    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 17:38:39.648129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648095    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-netns\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-run-netns\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.648259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648172    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovnkube-config\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.648259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648215    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-registration-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.648259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648228    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-netns\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-cni-multus\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648308    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-kubelet\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648352    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-kubelet\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648317    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-cni-multus\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648319    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-registration-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648406    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-lib-modules\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-ovn\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648495    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cc0998d-3386-4712-a688-acd2e7b04366-konnectivity-ca\") pod \"konnectivity-agent-7c6kr\" (UID: \"9cc0998d-3386-4712-a688-acd2e7b04366\") " pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-socket-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-k8s-cni-cncf-io\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.648584    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:39.648599 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-sys\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648652    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-var-lib-kubelet\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648662    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-k8s-cni-cncf-io\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.648675    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:40.148631284 +0000 UTC m=+3.018674231 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648716    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-log-socket\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648723    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-socket-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6zsk\" (UniqueName: \"kubernetes.io/projected/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-kube-api-access-d6zsk\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgc5x\" (UniqueName: \"kubernetes.io/projected/197940f3-4a76-4c8f-bf3f-c4adcf446604-kube-api-access-xgc5x\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648804    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-hosts-file\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648829    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-modprobe-d\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a154ec-3375-41ce-a3a3-55664240f6bb-tmp\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648916    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-hosts-file\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.648991    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-system-cni-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx2hz\" (UniqueName: \"kubernetes.io/projected/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-kube-api-access-jx2hz\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649068    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.649208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649099    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/89278d92-d875-487f-8fdb-6256db682130-iptables-alerter-script\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649136    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649153    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-system-cni-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649162    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k8l6\" (UniqueName: \"kubernetes.io/projected/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-kube-api-access-5k8l6\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cnibin\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649250    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-sys-fs\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649301    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39334c02-daa5-439d-9780-386add54e04b-cni-binary-copy\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-etc-kubernetes\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649345    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-systemd\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/59f9e5b9-528f-4400-a787-3c339153c3b4-dbus\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-etc-kubernetes\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-cni-bin\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649476    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-multus-certs\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649491    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-sys-fs\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.650043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-tmp-dir\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649538    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysconfig\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649573    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-run-multus-certs\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649576    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysctl-conf\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649616    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-kubelet\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649643    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-env-overrides\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/39334c02-daa5-439d-9780-386add54e04b-multus-daemon-config\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvmfb\" (UniqueName: \"kubernetes.io/projected/08a154ec-3375-41ce-a3a3-55664240f6bb-kube-api-access-gvmfb\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-host-var-lib-cni-bin\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649807    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/59f9e5b9-528f-4400-a787-3c339153c3b4-dbus\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-tmp-dir\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649900    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-hostroot\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649952    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649990    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-kubernetes\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-hostroot\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.649999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/89278d92-d875-487f-8fdb-6256db682130-iptables-alerter-script\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.650923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650045    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-var-lib-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650079    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovn-node-metrics-cert\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650110    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-system-cni-dir\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650138    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-socket-dir-parent\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650166    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlzlw\" (UniqueName: \"kubernetes.io/projected/39334c02-daa5-439d-9780-386add54e04b-kube-api-access-dlzlw\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650196    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-run\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650215    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-host\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-serviceca\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650237    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-socket-dir-parent\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-slash\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650304    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-node-log\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39334c02-daa5-439d-9780-386add54e04b-cni-binary-copy\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650345    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89278d92-d875-487f-8fdb-6256db682130-host-slash\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650389    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-conf-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/89278d92-d875-487f-8fdb-6256db682130-host-slash\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-etc-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650477    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-conf-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.651890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/59f9e5b9-528f-4400-a787-3c339153c3b4-kubelet-config\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-device-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650642    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/59f9e5b9-528f-4400-a787-3c339153c3b4-kubelet-config\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tbqp\" (UniqueName: \"kubernetes.io/projected/c6b182ac-212d-4b50-8cfc-36909e6f8d44-kube-api-access-8tbqp\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650674    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-device-dir\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650715    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-cni-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-tuned\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650774    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-multus-cni-dir\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650793    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cc0998d-3386-4712-a688-acd2e7b04366-agent-certs\") pod \"konnectivity-agent-7c6kr\" (UID: \"9cc0998d-3386-4712-a688-acd2e7b04366\") " pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/39334c02-daa5-439d-9780-386add54e04b-multus-daemon-config\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-cnibin\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-host\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.650980    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-systemd-units\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.650999    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651016    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-systemd\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.652770 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.651096    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:40.151046771 +0000 UTC m=+3.021089696 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651080    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39334c02-daa5-439d-9780-386add54e04b-cnibin\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-cni-netd\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651196    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovnkube-script-lib\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651225    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcct4\" (UniqueName: \"kubernetes.io/projected/89278d92-d875-487f-8fdb-6256db682130-kube-api-access-pcct4\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-etc-selinux\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651358    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysctl-d\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651392    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-cni-bin\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c6b182ac-212d-4b50-8cfc-36909e6f8d44-etc-selinux\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.653769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.651407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4n5d\" (UniqueName: \"kubernetes.io/projected/4a5cb5b0-f114-48c1-8859-2b9f267739a5-kube-api-access-b4n5d\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.662984 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.662841    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 17:38:39.663666 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.663644    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:39.663763 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.663671    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:39.663763 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.663687    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:39.663763 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:39.663749    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:40.163727051 +0000 UTC m=+3.033769978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:39.668075 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.667837    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcct4\" (UniqueName: \"kubernetes.io/projected/89278d92-d875-487f-8fdb-6256db682130-kube-api-access-pcct4\") pod \"iptables-alerter-9x8c4\" (UID: \"89278d92-d875-487f-8fdb-6256db682130\") " pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.668075 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.668022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tbqp\" (UniqueName: \"kubernetes.io/projected/c6b182ac-212d-4b50-8cfc-36909e6f8d44-kube-api-access-8tbqp\") pod \"aws-ebs-csi-driver-node-6ljw9\" (UID: \"c6b182ac-212d-4b50-8cfc-36909e6f8d44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.668269 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.668249    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgc5x\" (UniqueName: \"kubernetes.io/projected/197940f3-4a76-4c8f-bf3f-c4adcf446604-kube-api-access-xgc5x\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:39.668883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.668855    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlzlw\" (UniqueName: \"kubernetes.io/projected/39334c02-daa5-439d-9780-386add54e04b-kube-api-access-dlzlw\") pod \"multus-5ct78\" (UID: \"39334c02-daa5-439d-9780-386add54e04b\") " pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.668975 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.668862    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx2hz\" (UniqueName: \"kubernetes.io/projected/783d55b1-3f78-4435-9d56-cb60f6f7c2f3-kube-api-access-jx2hz\") pod \"node-resolver-vv7sb\" (UID: \"783d55b1-3f78-4435-9d56-cb60f6f7c2f3\") " pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.744654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.744592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"6a2ad4dde9645228f8d73db0210fa56c","Type":"ContainerStarted","Data":"da3134995bd6a89e2a080d012dcd99860e60eaec1fedb77c3676768cc993d6e4"}
Sep 11 17:38:39.745854 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.745822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal" event={"ID":"8c6603c74816e2e6b08fd8fe02fdf018","Type":"ContainerStarted","Data":"d585f6b537cc8f5c3b8158a2a7c45321a68955e61b24209c824c5f67e80ad114"}
Sep 11 17:38:39.752185 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-sys\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752281 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752191    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-var-lib-kubelet\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752281 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752208    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-log-socket\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.752281 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6zsk\" (UniqueName: \"kubernetes.io/projected/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-kube-api-access-d6zsk\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.752281 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752266    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-log-socket\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.752527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752296    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-sys\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752485    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-var-lib-kubelet\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752485    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-modprobe-d\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a154ec-3375-41ce-a3a3-55664240f6bb-tmp\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.752687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.752687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752644    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-modprobe-d\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752670    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752752    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k8l6\" (UniqueName: \"kubernetes.io/projected/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-kube-api-access-5k8l6\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752779    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cnibin\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752811    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-systemd\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752860    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cnibin\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752871    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752896    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752923    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysconfig\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.752952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752947    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysctl-conf\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-kubelet\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.752998    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-env-overrides\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvmfb\" (UniqueName: \"kubernetes.io/projected/08a154ec-3375-41ce-a3a3-55664240f6bb-kube-api-access-gvmfb\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753041    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysconfig\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753053    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-kubernetes\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-var-lib-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753104    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovn-node-metrics-cert\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-system-cni-dir\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-run\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-host\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753201    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-serviceca\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753197    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753241    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-var-lib-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysctl-conf\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.753469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753457    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-kubelet\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753501    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-kubernetes\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753563    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-run\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753561    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-cni-binary-copy\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753562    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-slash\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753602    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-slash\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753612    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-system-cni-dir\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753630    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-node-log\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753661    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-host\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753665    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-etc-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753670    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-systemd\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753695    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-tuned\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753701    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-node-log\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753722    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cc0998d-3386-4712-a688-acd2e7b04366-agent-certs\") pod \"konnectivity-agent-7c6kr\" (UID: \"9cc0998d-3386-4712-a688-acd2e7b04366\") " pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753758    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-etc-openvswitch\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-host\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.754235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753884    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-systemd-units\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-systemd\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753936    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753968    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-cni-netd\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.753994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovnkube-script-lib\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysctl-d\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754049    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-env-overrides\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-cni-bin\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4n5d\" (UniqueName: \"kubernetes.io/projected/4a5cb5b0-f114-48c1-8859-2b9f267739a5-kube-api-access-b4n5d\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754104    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-serviceca\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-run-netns\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754116    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-run-ovn-kubernetes\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovnkube-config\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754181    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-cni-bin\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754217    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-lib-modules\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754240    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-cni-netd\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754246    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-ovn\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754295    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-ovn\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754325    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cc0998d-3386-4712-a688-acd2e7b04366-konnectivity-ca\") pod \"konnectivity-agent-7c6kr\" (UID: \"9cc0998d-3386-4712-a688-acd2e7b04366\") " pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-systemd-units\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754582    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-host\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754688    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-run-systemd\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754816    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovnkube-script-lib\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754821    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-sysctl-d\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/4a5cb5b0-f114-48c1-8859-2b9f267739a5-host-run-netns\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754913    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9cc0998d-3386-4712-a688-acd2e7b04366-konnectivity-ca\") pod \"konnectivity-agent-7c6kr\" (UID: \"9cc0998d-3386-4712-a688-acd2e7b04366\") " pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.754969    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08a154ec-3375-41ce-a3a3-55664240f6bb-lib-modules\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.755749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.755299    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovnkube-config\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.756105 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.755835    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08a154ec-3375-41ce-a3a3-55664240f6bb-tmp\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.756499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.756473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/08a154ec-3375-41ce-a3a3-55664240f6bb-etc-tuned\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.756679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.756649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9cc0998d-3386-4712-a688-acd2e7b04366-agent-certs\") pod \"konnectivity-agent-7c6kr\" (UID: \"9cc0998d-3386-4712-a688-acd2e7b04366\") " pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.756860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.756843    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/4a5cb5b0-f114-48c1-8859-2b9f267739a5-ovn-node-metrics-cert\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.769489 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.769421    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k8l6\" (UniqueName: \"kubernetes.io/projected/f6fe38a1-c4b8-4cdd-8156-27a378e522f7-kube-api-access-5k8l6\") pod \"node-ca-xx9jz\" (UID: \"f6fe38a1-c4b8-4cdd-8156-27a378e522f7\") " pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.769676 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.769656    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6zsk\" (UniqueName: \"kubernetes.io/projected/9d27ca5d-8fc8-47a1-b7e0-af23330951d6-kube-api-access-d6zsk\") pod \"multus-additional-cni-plugins-pmgpd\" (UID: \"9d27ca5d-8fc8-47a1-b7e0-af23330951d6\") " pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:39.770705 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.770681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvmfb\" (UniqueName: \"kubernetes.io/projected/08a154ec-3375-41ce-a3a3-55664240f6bb-kube-api-access-gvmfb\") pod \"tuned-gvj4q\" (UID: \"08a154ec-3375-41ce-a3a3-55664240f6bb\") " pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.770791 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.770781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4n5d\" (UniqueName: \"kubernetes.io/projected/4a5cb5b0-f114-48c1-8859-2b9f267739a5-kube-api-access-b4n5d\") pod \"ovnkube-node-nwqjv\" (UID: \"4a5cb5b0-f114-48c1-8859-2b9f267739a5\") " pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.837287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.837258    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9x8c4"
Sep 11 17:38:39.846978 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.846952    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9"
Sep 11 17:38:39.858050 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.858018    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vv7sb"
Sep 11 17:38:39.864949 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.864931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5ct78"
Sep 11 17:38:39.871843 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.871817    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:38:39.879828 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.879810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:38:39.887684 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.887667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-gvj4q"
Sep 11 17:38:39.903503 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.903481    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xx9jz"
Sep 11 17:38:39.910342 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:39.910317    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pmgpd"
Sep 11 17:38:40.157110 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.157012    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:40.157110 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.157077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:40.157328 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.157184    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:40.157328 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.157268    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:41.157249892 +0000 UTC m=+4.027292853 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:40.157328 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.157194    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:40.157502 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.157374    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:41.157344262 +0000 UTC m=+4.027387195 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:40.258145 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.258114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:40.258317 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.258298    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:40.258378 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.258327    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:40.258378 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.258341    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:40.258483 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.258413    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:41.258395171 +0000 UTC m=+4.128438102 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:40.587533 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.587493    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 17:33:39 +0000 UTC" deadline="2028-06-03 05:55:07.721929238 +0000 UTC"
Sep 11 17:38:40.587533 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.587526    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15132h16m27.134406898s"
Sep 11 17:38:40.740697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.740667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:40.740861 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:40.740667    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:40.740861 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.740781    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:40.740861 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:40.740851    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:40.790845 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.790801    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5cb5b0_f114_48c1_8859_2b9f267739a5.slice/crio-1a1b154b0f555cb6996614e032aefc523f51370dcc1caf4321492d29f54eb7e9 WatchSource:0}: Error finding container 1a1b154b0f555cb6996614e032aefc523f51370dcc1caf4321492d29f54eb7e9: Status 404 returned error can't find the container with id 1a1b154b0f555cb6996614e032aefc523f51370dcc1caf4321492d29f54eb7e9
Sep 11 17:38:40.791551 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.791526    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a154ec_3375_41ce_a3a3_55664240f6bb.slice/crio-94c1b0cd82de6e24b5954ae6286a6a334668b2453f3e6ee67acf178069725839 WatchSource:0}: Error finding container 94c1b0cd82de6e24b5954ae6286a6a334668b2453f3e6ee67acf178069725839: Status 404 returned error can't find the container with id 94c1b0cd82de6e24b5954ae6286a6a334668b2453f3e6ee67acf178069725839
Sep 11 17:38:40.793133 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.793087    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d27ca5d_8fc8_47a1_b7e0_af23330951d6.slice/crio-16271f54b65fdb2f3b017b880bea0dd6981244af149ca15d9793a18332e74d23 WatchSource:0}: Error finding container 16271f54b65fdb2f3b017b880bea0dd6981244af149ca15d9793a18332e74d23: Status 404 returned error can't find the container with id 16271f54b65fdb2f3b017b880bea0dd6981244af149ca15d9793a18332e74d23
Sep 11 17:38:40.794420 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.794378    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc6b182ac_212d_4b50_8cfc_36909e6f8d44.slice/crio-0ab4f634c317bb76ac2a5951b2eecaa6484a99ae9995206f5d947eb2b5851496 WatchSource:0}: Error finding container 0ab4f634c317bb76ac2a5951b2eecaa6484a99ae9995206f5d947eb2b5851496: Status 404 returned error can't find the container with id 0ab4f634c317bb76ac2a5951b2eecaa6484a99ae9995206f5d947eb2b5851496
Sep 11 17:38:40.795807 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.795652    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6fe38a1_c4b8_4cdd_8156_27a378e522f7.slice/crio-d9d4c8a1d5fe536fe09d35abb5ce1fcb3e97ece82aee717f6b296af8f71064c3 WatchSource:0}: Error finding container d9d4c8a1d5fe536fe09d35abb5ce1fcb3e97ece82aee717f6b296af8f71064c3: Status 404 returned error can't find the container with id d9d4c8a1d5fe536fe09d35abb5ce1fcb3e97ece82aee717f6b296af8f71064c3
Sep 11 17:38:40.797303 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.797276    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc0998d_3386_4712_a688_acd2e7b04366.slice/crio-d27b1a532e3ff5361343e68781524232190ff1a4c601b7b7d80167816893e02a WatchSource:0}: Error finding container d27b1a532e3ff5361343e68781524232190ff1a4c601b7b7d80167816893e02a: Status 404 returned error can't find the container with id d27b1a532e3ff5361343e68781524232190ff1a4c601b7b7d80167816893e02a
Sep 11 17:38:40.808304 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:38:40.808233    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89278d92_d875_487f_8fdb_6256db682130.slice/crio-f525a7fbb8ebf0669d162b17a00e1f4a70ee2d27facd6125b82234c260bd2196 WatchSource:0}: Error finding container f525a7fbb8ebf0669d162b17a00e1f4a70ee2d27facd6125b82234c260bd2196: Status 404 returned error can't find the container with id f525a7fbb8ebf0669d162b17a00e1f4a70ee2d27facd6125b82234c260bd2196
Sep 11 17:38:41.165472 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.165438    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:41.165607 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.165498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:41.165607 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.165524    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:41.165607 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.165584    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:43.165568406 +0000 UTC m=+6.035611330 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:41.165792 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.165613    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:41.165792 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.165658    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:43.165646396 +0000 UTC m=+6.035689321 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:41.265907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.265865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:41.266095 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.266048    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:41.266095 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.266073    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:41.266095 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.266087    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:41.266255 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.266150    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:43.266129586 +0000 UTC m=+6.136172516 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:41.588058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.587995    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 17:33:39 +0000 UTC" deadline="2028-05-27 00:48:18.89373025 +0000 UTC"
Sep 11 17:38:41.588058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.588054    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14959h9m37.30567981s"
Sep 11 17:38:41.740979 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.740727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:41.741139 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:41.741083    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:41.775975 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.775939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5ct78" event={"ID":"39334c02-daa5-439d-9780-386add54e04b","Type":"ContainerStarted","Data":"d7c56f124ee47ac03b63af0d6b95994320b771fb88ef1b450272e84debc1f3c8"}
Sep 11 17:38:41.780834 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.780761    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vv7sb" event={"ID":"783d55b1-3f78-4435-9d56-cb60f6f7c2f3","Type":"ContainerStarted","Data":"02c592c1ad32d0d3d50ac009fdb0042d469380fb286729a91cfc00a2b6b3af25"}
Sep 11 17:38:41.804530 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.804416    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9" event={"ID":"c6b182ac-212d-4b50-8cfc-36909e6f8d44","Type":"ContainerStarted","Data":"0ab4f634c317bb76ac2a5951b2eecaa6484a99ae9995206f5d947eb2b5851496"}
Sep 11 17:38:41.822544 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.822513    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gvj4q" event={"ID":"08a154ec-3375-41ce-a3a3-55664240f6bb","Type":"ContainerStarted","Data":"94c1b0cd82de6e24b5954ae6286a6a334668b2453f3e6ee67acf178069725839"}
Sep 11 17:38:41.834783 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.834752    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"1a1b154b0f555cb6996614e032aefc523f51370dcc1caf4321492d29f54eb7e9"}
Sep 11 17:38:41.849195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.849125    2582 generic.go:356] "Generic (PLEG): container finished" podID="6a2ad4dde9645228f8d73db0210fa56c" containerID="07b24091a31e567ccdc149c59ae309b4624ba2283a4b4c5aa535a1d3d4529c76" exitCode=0
Sep 11 17:38:41.849312 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.849211    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"6a2ad4dde9645228f8d73db0210fa56c","Type":"ContainerDied","Data":"07b24091a31e567ccdc149c59ae309b4624ba2283a4b4c5aa535a1d3d4529c76"}
Sep 11 17:38:41.858010 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.857983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal" event={"ID":"8c6603c74816e2e6b08fd8fe02fdf018","Type":"ContainerStarted","Data":"58a4c328489f591d92f4331d56395891649cc2d9eee1bfd72eba7dd93815b5b9"}
Sep 11 17:38:41.875360 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.875244    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9x8c4" event={"ID":"89278d92-d875-487f-8fdb-6256db682130","Type":"ContainerStarted","Data":"f525a7fbb8ebf0669d162b17a00e1f4a70ee2d27facd6125b82234c260bd2196"}
Sep 11 17:38:41.886104 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.886048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7c6kr" event={"ID":"9cc0998d-3386-4712-a688-acd2e7b04366","Type":"ContainerStarted","Data":"d27b1a532e3ff5361343e68781524232190ff1a4c601b7b7d80167816893e02a"}
Sep 11 17:38:41.900260 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.900183    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xx9jz" event={"ID":"f6fe38a1-c4b8-4cdd-8156-27a378e522f7","Type":"ContainerStarted","Data":"d9d4c8a1d5fe536fe09d35abb5ce1fcb3e97ece82aee717f6b296af8f71064c3"}
Sep 11 17:38:41.905661 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:41.905589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerStarted","Data":"16271f54b65fdb2f3b017b880bea0dd6981244af149ca15d9793a18332e74d23"}
Sep 11 17:38:42.741317 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:42.740662    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:42.741317 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:42.740804    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:42.741317 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:42.741161    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:42.741317 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:42.741254    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:42.929361 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:42.929145    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" event={"ID":"6a2ad4dde9645228f8d73db0210fa56c","Type":"ContainerStarted","Data":"6754e27b5b107ceb5ed7999504bb10cba54199a47c51e024911d3f53d0a242a2"}
Sep 11 17:38:42.942687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:42.942496    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-237.ec2.internal" podStartSLOduration=4.942475345 podStartE2EDuration="4.942475345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:41.884086913 +0000 UTC m=+4.754129874" watchObservedRunningTime="2026-09-11 17:38:42.942475345 +0000 UTC m=+5.812518294"
Sep 11 17:38:43.182149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:43.181771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:43.182149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:43.181827    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:43.182149 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.181969    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:43.182149 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.181996    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:43.182149 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.182027    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:47.182008235 +0000 UTC m=+10.052051161 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:43.182149 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.182047    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:47.182034944 +0000 UTC m=+10.052077871 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:43.283272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:43.283161    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:43.283466 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.283401    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:43.283466 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.283429    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:43.283466 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.283442    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:43.283632 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.283499    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:47.283481314 +0000 UTC m=+10.153524239 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:43.742054 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:43.742022    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:43.742526 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:43.742155    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:44.740343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:44.740310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:44.740569 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:44.740470    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:44.740569 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:44.740310    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:44.740681 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:44.740594    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:45.742868 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:45.740766    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:45.742868 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:45.740891    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:46.740944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:46.740910    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:46.741149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:46.740960    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:46.741149 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:46.741060    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:46.741313 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:46.741168    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:47.219695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:47.219601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:47.219695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:47.219687    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:47.220138 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.219799    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:47.220138 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.219854    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:55.219841574 +0000 UTC m=+18.089884500 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:47.220138 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.219799    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:47.220138 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.219889    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:38:55.21988007 +0000 UTC m=+18.089923007 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:47.320962 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:47.320381    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:47.320962 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.320635    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:47.320962 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.320657    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:47.320962 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.320671    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:47.320962 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.320742    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:38:55.320721745 +0000 UTC m=+18.190764692 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:47.741837 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:47.741523    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:47.741837 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:47.741627    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:48.740828 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:48.740760    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:48.741243 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:48.740887    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:48.741243 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:48.741210    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:48.741369 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:48.741318    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:49.740739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:49.740702    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:49.740895 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:49.740812    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:50.740809 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:50.740768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:50.740968 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:50.740788    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:50.740968 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:50.740921    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:50.740968 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:50.740950    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:51.740480 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:51.740448    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:51.740657 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:51.740569    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:52.740863 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:52.740824    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:52.741263 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:52.740870    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:52.741263 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:52.740972    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:52.741263 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:52.741078    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:53.740155 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:53.740113    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:53.740318 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:53.740257    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:54.740281 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:54.740242    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:54.740756 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:54.740260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:54.740756 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:54.740401    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:54.740756 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:54.740484    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:55.278560 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:55.278517    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:55.278754 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:55.278577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:55.278754 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.278684    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:55.278849 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.278756    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:11.278741576 +0000 UTC m=+34.148784503 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:38:55.278849 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.278684    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:55.278849 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.278822    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:11.278809588 +0000 UTC m=+34.148852527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:38:55.378909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:55.378870    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:55.379092 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.379069    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:38:55.379152 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.379094    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:38:55.379152 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.379107    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:55.379251 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.379176    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:11.379157209 +0000 UTC m=+34.249200151 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:38:55.740869 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:55.740839    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:55.741330 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:55.740988    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:56.740254 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:56.740221    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:56.740254 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:56.740255    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:56.740492 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:56.740389    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:56.740587 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:56.740553    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:57.741620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:57.741590    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:57.742012 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:57.741679    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:58.740737 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.740514    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:38:58.740860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.740582    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:38:58.740923 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:58.740880    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:38:58.740967 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:58.740920    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:38:58.963227 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.963199    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vv7sb" event={"ID":"783d55b1-3f78-4435-9d56-cb60f6f7c2f3","Type":"ContainerStarted","Data":"c4d5c41d0ce8db025dcb92a8383ab9872baed92419e55c58883948e1bb808dd9"}
Sep 11 17:38:58.965166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.964941    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9" event={"ID":"c6b182ac-212d-4b50-8cfc-36909e6f8d44","Type":"ContainerStarted","Data":"864273c0e8a3f867938428c56b8d441cab6e6934f3c11e2bfb39b3c81d70e3e4"}
Sep 11 17:38:58.966585 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.966558    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-gvj4q" event={"ID":"08a154ec-3375-41ce-a3a3-55664240f6bb","Type":"ContainerStarted","Data":"53f382d324829e18217e73189535d38547b0dd265049b170433740636a6c9e1a"}
Sep 11 17:38:58.969690 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.969662    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"036c8c4fa38c57cfcf5e5df18ee2eed6d8473329692e657ce37911f5c2aae30f"}
Sep 11 17:38:58.969777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.969725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"23aad40c990cd1c55e6b80151ff1e3e2a7054274505e9ea89bf2d54b764aa76d"}
Sep 11 17:38:58.969777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.969771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"88a49bb139062b64711ad322543618fdda58618770d9e46a70de81e6143f6450"}
Sep 11 17:38:58.969896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.969813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"e0c1537bb9cf537fa2ff47f1bb3548680eec3666021cd3a7d7fd2551b877888e"}
Sep 11 17:38:58.969896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.969847    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"66176d4c74b6c0a1c4b3347ebd9d93d7c07beeddff2cb63b1374f27fe49b698c"}
Sep 11 17:38:58.971101 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.971077    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7c6kr" event={"ID":"9cc0998d-3386-4712-a688-acd2e7b04366","Type":"ContainerStarted","Data":"f80569c414f82891294833fce23babdac10aec61e8bb0db20941fa8e23898d60"}
Sep 11 17:38:58.972794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.972775    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xx9jz" event={"ID":"f6fe38a1-c4b8-4cdd-8156-27a378e522f7","Type":"ContainerStarted","Data":"a7939fa2d9e94ab3b37f329a40502164718ce59f6ea412e638949a783623b05a"}
Sep 11 17:38:58.974499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.974475    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d27ca5d-8fc8-47a1-b7e0-af23330951d6" containerID="2bb0807035d5591fe84731ff3bab8d4ee0531943383df62a634c7b4d369f1139" exitCode=0
Sep 11 17:38:58.974592 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.974556    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerDied","Data":"2bb0807035d5591fe84731ff3bab8d4ee0531943383df62a634c7b4d369f1139"}
Sep 11 17:38:58.977734 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.977669    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-237.ec2.internal" podStartSLOduration=20.97765185 podStartE2EDuration="20.97765185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:42.948517601 +0000 UTC m=+5.818560546" watchObservedRunningTime="2026-09-11 17:38:58.97765185 +0000 UTC m=+21.847694795"
Sep 11 17:38:58.978425 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.978083    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vv7sb" podStartSLOduration=3.388531767 podStartE2EDuration="20.978068766s" totalImagesPullingTime="17.589536999s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:58.977803122 +0000 UTC m=+21.847846065" watchObservedRunningTime="2026-09-11 17:38:58.978068766 +0000 UTC m=+21.848111715"
Sep 11 17:38:58.979021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.978990    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5ct78" event={"ID":"39334c02-daa5-439d-9780-386add54e04b","Type":"ContainerStarted","Data":"94955005cadaccf0b251e82c34c18faf4e7c1d6acee74d664acdcc0209be2931"}
Sep 11 17:38:58.994859 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:58.994810    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-gvj4q" podStartSLOduration=3.394238911 podStartE2EDuration="20.994791019s" totalImagesPullingTime="17.600552108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:58.994603627 +0000 UTC m=+21.864646573" watchObservedRunningTime="2026-09-11 17:38:58.994791019 +0000 UTC m=+21.864833966"
Sep 11 17:38:59.008942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.008901    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xx9jz" podStartSLOduration=3.41604916 podStartE2EDuration="21.008887077s" totalImagesPullingTime="17.592837917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:59.008744301 +0000 UTC m=+21.878787249" watchObservedRunningTime="2026-09-11 17:38:59.008887077 +0000 UTC m=+21.878930021"
Sep 11 17:38:59.070086 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.069984    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7c6kr" podStartSLOduration=7.54411622 podStartE2EDuration="21.069962306s" totalImagesPullingTime="13.525846086s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:59.069593008 +0000 UTC m=+21.939635955" watchObservedRunningTime="2026-09-11 17:38:59.069962306 +0000 UTC m=+21.940005251"
Sep 11 17:38:59.690662 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.690636    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 17:38:59.740142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.740108    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:38:59.740292 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:38:59.740233    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:38:59.753728 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.753626    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T17:38:59.690658886Z","UUID":"f40f66de-9154-4cc6-bdf4-5c03f5a03328","Handler":null,"Name":"","Endpoint":""}
Sep 11 17:38:59.755487 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.755465    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 17:38:59.755610 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.755495    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 17:38:59.983842 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.983810    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9" event={"ID":"c6b182ac-212d-4b50-8cfc-36909e6f8d44","Type":"ContainerStarted","Data":"1737f2ecf764e90bfe3c2f59af1e477858994aa2ccc5150261073e54b8193c45"}
Sep 11 17:38:59.987411 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.987384    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"c878010c7e1efc8c80654535e311d0f3033faa92c67e682fb7ff4ef10e9a4a47"}
Sep 11 17:38:59.989087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:38:59.989025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9x8c4" event={"ID":"89278d92-d875-487f-8fdb-6256db682130","Type":"ContainerStarted","Data":"b73d7490d6e877aa5a8cb11ec0cdb563d7d9e6d28f0afedbd7ff8ed5b9c807f1"}
Sep 11 17:39:00.004973 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:00.004930    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9x8c4" podStartSLOduration=4.423541806 podStartE2EDuration="22.004916737s" totalImagesPullingTime="17.581374931s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:00.004698985 +0000 UTC m=+22.874741933" watchObservedRunningTime="2026-09-11 17:39:00.004916737 +0000 UTC m=+22.874959683"
Sep 11 17:39:00.005155 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:00.005134    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5ct78" podStartSLOduration=4.213442328 podStartE2EDuration="22.005128183s" totalImagesPullingTime="17.791685855s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:38:59.096581335 +0000 UTC m=+21.966624306" watchObservedRunningTime="2026-09-11 17:39:00.005128183 +0000 UTC m=+22.875171193"
Sep 11 17:39:00.740861 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:00.740837    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:00.741007 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:00.740955    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:39:00.741073 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:00.741056    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:00.741173 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:00.741154    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:39:00.993966 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:00.993927    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9" event={"ID":"c6b182ac-212d-4b50-8cfc-36909e6f8d44","Type":"ContainerStarted","Data":"66eddce3cb02cf70e059cdc22b7c6b34ebcfa18fe24c8cbc90f7573d6756f027"}
Sep 11 17:39:01.012789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:01.012749    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-6ljw9" podStartSLOduration=3.456725381 podStartE2EDuration="23.012735891s" totalImagesPullingTime="19.55601051s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:01.012637741 +0000 UTC m=+23.882680688" watchObservedRunningTime="2026-09-11 17:39:01.012735891 +0000 UTC m=+23.882778836"
Sep 11 17:39:01.740456 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:01.740419    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:01.740607 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:01.740551    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:39:02.000766 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:02.000682    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"8507d83c8b140f6f70484d455bf8c287c89b4c0856b2305241b0265b86dd249c"}
Sep 11 17:39:02.301247 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:02.301161    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:39:02.301877 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:02.301853    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:39:02.740670 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:02.740642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:02.740829 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:02.740642    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:02.740829 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:02.740756    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:39:02.740949 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:02.740840    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:39:03.003382 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:03.003287    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:39:03.003848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:03.003828    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7c6kr"
Sep 11 17:39:03.740755 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:03.740599    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:03.742101 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:03.741138    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:39:04.006829 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:04.006803    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d27ca5d-8fc8-47a1-b7e0-af23330951d6" containerID="9e2565264cf047e7912cbad370c29898221296b58e366efdc8efee3c07ed4733" exitCode=0
Sep 11 17:39:04.007647 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:04.006891    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerDied","Data":"9e2565264cf047e7912cbad370c29898221296b58e366efdc8efee3c07ed4733"}
Sep 11 17:39:04.010730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:04.010707    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" event={"ID":"4a5cb5b0-f114-48c1-8859-2b9f267739a5","Type":"ContainerStarted","Data":"30ef77a2ceab0a96bb71523ded01a9e7bfff7e068edb69d01c08995af3f37628"}
Sep 11 17:39:04.052048 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:04.052006    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv" podStartSLOduration=8.427896948 podStartE2EDuration="26.051991294s" totalImagesPullingTime="17.624094346s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:04.051270533 +0000 UTC m=+26.921313479" watchObservedRunningTime="2026-09-11 17:39:04.051991294 +0000 UTC m=+26.922034239"
Sep 11 17:39:04.741003 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:04.740957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:04.741003 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:04.740981    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:04.741263 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:04.741127    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:39:04.741263 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:04.741217    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:39:05.013885 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.013819    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:39:05.014284 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.013901    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:39:05.014284 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.013947    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:39:05.028719 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.028691    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:39:05.028974 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.028953    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:39:05.494093 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.493888    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7h22x"]
Sep 11 17:39:05.494251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.494215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:05.494338 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:05.494318    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:39:05.497260 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.497236    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cgzbn"]
Sep 11 17:39:05.497409 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.497324    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:05.497471 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:05.497422    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:39:05.509308 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.509284    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rr858"]
Sep 11 17:39:05.509420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:05.509394    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:05.509482 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:05.509466    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:39:06.017235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:06.017207    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d27ca5d-8fc8-47a1-b7e0-af23330951d6" containerID="9cfaf0f39c30bbc4395e27fe0ed2120d92e496b9ffe5c71b5ac88b3b86a54363" exitCode=0
Sep 11 17:39:06.017640 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:06.017270    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerDied","Data":"9cfaf0f39c30bbc4395e27fe0ed2120d92e496b9ffe5c71b5ac88b3b86a54363"}
Sep 11 17:39:07.741534 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:07.741502    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:07.741971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:07.741547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:07.741971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:07.741608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:07.741971 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:07.741637    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:39:07.741971 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:07.741711    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:39:07.741971 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:07.741814    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:39:08.024031 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:08.023998    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d27ca5d-8fc8-47a1-b7e0-af23330951d6" containerID="852449a8037ef083b6e5036537513e085f69d3fdab1a2b9981d46fc166220205" exitCode=0
Sep 11 17:39:08.024177 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:08.024079    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerDied","Data":"852449a8037ef083b6e5036537513e085f69d3fdab1a2b9981d46fc166220205"}
Sep 11 17:39:09.740694 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:09.740655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:09.740694 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:09.740685    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:09.741266 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:09.740655    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:09.741266 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:09.740746    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rr858" podUID="c9347efe-d030-4e8f-b60f-9929b060c87a"
Sep 11 17:39:09.741266 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:09.740844    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-cgzbn" podUID="59f9e5b9-528f-4400-a787-3c339153c3b4"
Sep 11 17:39:09.741266 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:09.740919    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-7h22x" podUID="197940f3-4a76-4c8f-bf3f-c4adcf446604"
Sep 11 17:39:11.299053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.298846    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:11.299476 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.299081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:11.299476 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.299013    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:11.299652 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.299627    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:11.299782 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.299689    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret podName:59f9e5b9-528f-4400-a787-3c339153c3b4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.299664335 +0000 UTC m=+66.169707275 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret") pod "global-pull-secret-syncer-cgzbn" (UID: "59f9e5b9-528f-4400-a787-3c339153c3b4") : object "kube-system"/"original-pull-secret" not registered
Sep 11 17:39:11.299782 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.299720    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs podName:197940f3-4a76-4c8f-bf3f-c4adcf446604 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.299710009 +0000 UTC m=+66.169752934 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs") pod "network-metrics-daemon-7h22x" (UID: "197940f3-4a76-4c8f-bf3f-c4adcf446604") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 17:39:11.399837 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.399808    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:11.400018 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.399999    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 17:39:11.400081 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.400023    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 17:39:11.400081 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.400037    2582 projected.go:196] Error preparing data for projected volume kube-api-access-rc44s for pod openshift-network-diagnostics/network-check-target-rr858: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:11.400162 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.400097    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s podName:c9347efe-d030-4e8f-b60f-9929b060c87a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.400077174 +0000 UTC m=+66.270120099 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rc44s" (UniqueName: "kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s") pod "network-check-target-rr858" (UID: "c9347efe-d030-4e8f-b60f-9929b060c87a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 17:39:11.437038 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.437011    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-237.ec2.internal" event="NodeReady"
Sep 11 17:39:11.437177 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.437151    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 17:39:11.472217 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.472189    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8bcc4bd74-qz5zs"]
Sep 11 17:39:11.475985 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.475967    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.478294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.477990    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 11 17:39:11.478294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.478089    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 11 17:39:11.478294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.478090    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2cprr\""
Sep 11 17:39:11.478294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.478141    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 11 17:39:11.484079 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.484060    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 11 17:39:11.486075 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.486055    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"]
Sep 11 17:39:11.491324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.491304    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-ht8lt"]
Sep 11 17:39:11.491797 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.491535    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"
Sep 11 17:39:11.493343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.493298    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-zrt7p\""
Sep 11 17:39:11.494051 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.494026    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.494292 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.494277    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.496058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.495090    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"]
Sep 11 17:39:11.496058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.495215    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.497577 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.497558    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.497854 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.497834    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 11 17:39:11.497942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.497573    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 11 17:39:11.498165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.498145    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2hjz5\""
Sep 11 17:39:11.498294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.498237    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.498999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.498959    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-bnb6l"]
Sep 11 17:39:11.499085 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.499008    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.502603 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.502585    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bcc4bd74-qz5zs"]
Sep 11 17:39:11.502683 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.502627    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-89b8b6fbd-68pzd"]
Sep 11 17:39:11.502803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.502784    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.503100 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.503081    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 11 17:39:11.506371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.506337    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"]
Sep 11 17:39:11.508298 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.508157    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.510839 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.510302    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.512827 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.512803    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 11 17:39:11.513309 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.513289    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 11 17:39:11.514695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.514674    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"]
Sep 11 17:39:11.514988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.514971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:11.516709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.516692    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kwmgn\""
Sep 11 17:39:11.516838 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.516821    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517079    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x7pnl\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517093    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517082    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517448    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517724    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517778    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.517967    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.518083    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.518405    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nhczd\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.518956    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.519035    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 11 17:39:11.519320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.519113    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 11 17:39:11.520970 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.520950    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 11 17:39:11.521440 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.521420    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-249wm\""
Sep 11 17:39:11.521548 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.521488    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 11 17:39:11.521637 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.521614    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 11 17:39:11.522161 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.522141    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"]
Sep 11 17:39:11.522979 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.522962    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.525012 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.524994    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.525613 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.525439    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ffkl5\""
Sep 11 17:39:11.525613 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.525506    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.525749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.525658    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 11 17:39:11.525749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.525674    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 11 17:39:11.525852 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.525445    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"]
Sep 11 17:39:11.525852 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.525678    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.528085 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.528063    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.529442 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.529397    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bgbb4\""
Sep 11 17:39:11.529739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.529558    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 11 17:39:11.529739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.529589    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.529739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.529639    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 11 17:39:11.530392 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.530295    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"]
Sep 11 17:39:11.530567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.530552    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"
Sep 11 17:39:11.532938 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.532922    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-48xvp\""
Sep 11 17:39:11.533412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.533192    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.534824 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.534255    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.535330 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.535298    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-545rn"]
Sep 11 17:39:11.535914 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.535870    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:11.538049 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.538029    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.538842 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.538825    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.539202 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.539184    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rxpjs\""
Sep 11 17:39:11.539305 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.539184    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 11 17:39:11.539542 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.539525    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zzp4k"]
Sep 11 17:39:11.539777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.539616    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.542197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.542179    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 17:39:11.542303 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.542286    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zb994\""
Sep 11 17:39:11.542623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.542604    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 17:39:11.543080 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543062    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"]
Sep 11 17:39:11.543164 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543098    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"]
Sep 11 17:39:11.543164 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543120    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"]
Sep 11 17:39:11.543164 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543141    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"]
Sep 11 17:39:11.543164 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543164    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-545rn"]
Sep 11 17:39:11.543379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.543379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543186    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"]
Sep 11 17:39:11.543379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543199    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"]
Sep 11 17:39:11.543379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543217    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-89b8b6fbd-68pzd"]
Sep 11 17:39:11.543379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543233    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-ht8lt"]
Sep 11 17:39:11.543379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.543248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"]
Sep 11 17:39:11.545998 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.545978    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 17:39:11.546258 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.546239    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 17:39:11.546537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.546518    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 17:39:11.546881 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.546864    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-66cbf\""
Sep 11 17:39:11.550030 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.550012    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zzp4k"]
Sep 11 17:39:11.553658 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.553636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-bnb6l"]
Sep 11 17:39:11.600674 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.600648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-bound-sa-token\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.600850 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.600823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-default-certificate\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.600955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.600868    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-ca-trust-extracted\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.600955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.600896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a040f9a-b722-4aa6-9cf3-371f02ee172a-tmp\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.601055 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.600963    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a040f9a-b722-4aa6-9cf3-371f02ee172a-service-ca-bundle\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.601055 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a040f9a-b722-4aa6-9cf3-371f02ee172a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.601130 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601052    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hsmz\" (UniqueName: \"kubernetes.io/projected/69bfb1df-6b20-422a-9264-bd89506ab8d4-kube-api-access-9hsmz\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.601130 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601077    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.601130 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601100    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.601130 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601122    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-trusted-ca\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-stats-auth\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73c2ef90-9bd5-4110-892d-296f12551de4-config\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601181    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z7jx\" (UniqueName: \"kubernetes.io/projected/7a040f9a-b722-4aa6-9cf3-371f02ee172a-kube-api-access-4z7jx\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601204    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a040f9a-b722-4aa6-9cf3-371f02ee172a-serving-cert\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601244    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/041f6f00-15f9-48cf-82a2-fe253ff1f70a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:11.601262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601258    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh57m\" (UniqueName: \"kubernetes.io/projected/a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb-kube-api-access-dh57m\") pod \"volume-data-source-validator-86695b7cc7-nvxnn\" (UID: \"a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601291    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7dcv\" (UniqueName: \"kubernetes.io/projected/cd176d7f-b655-4a6a-90d3-7df96b78de54-kube-api-access-n7dcv\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601317    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-installation-pull-secrets\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73c2ef90-9bd5-4110-892d-296f12551de4-serving-cert\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601452    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzvvh\" (UniqueName: \"kubernetes.io/projected/aaae7d8f-4e08-4d25-ad88-0253fb26e743-kube-api-access-gzvvh\") pod \"network-check-source-6f8b768455-2hq8s\" (UID: \"aaae7d8f-4e08-4d25-ad88-0253fb26e743\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601525    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd176d7f-b655-4a6a-90d3-7df96b78de54-config\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.601567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601560    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd176d7f-b655-4a6a-90d3-7df96b78de54-trusted-ca\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601588    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-certificates\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601617    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e471abfe-aaf5-45bb-8f10-9ec40578bf95-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601643    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shwq5\" (UniqueName: \"kubernetes.io/projected/73c2ef90-9bd5-4110-892d-296f12551de4-kube-api-access-shwq5\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-image-registry-private-configuration\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601739    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncv24\" (UniqueName: \"kubernetes.io/projected/e471abfe-aaf5-45bb-8f10-9ec40578bf95-kube-api-access-ncv24\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd176d7f-b655-4a6a-90d3-7df96b78de54-serving-cert\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601791    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e471abfe-aaf5-45bb-8f10-9ec40578bf95-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.601883 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601869    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7a040f9a-b722-4aa6-9cf3-371f02ee172a-snapshots\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.602238 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601896    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hlvh\" (UniqueName: \"kubernetes.io/projected/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-kube-api-access-2hlvh\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.602238 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.601913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fhrx\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-kube-api-access-9fhrx\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.688861 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.688830    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:11.702871 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.702836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a040f9a-b722-4aa6-9cf3-371f02ee172a-tmp\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703015 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.702885    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a040f9a-b722-4aa6-9cf3-371f02ee172a-service-ca-bundle\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703015 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.702919    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a040f9a-b722-4aa6-9cf3-371f02ee172a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703015 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.702943    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hsmz\" (UniqueName: \"kubernetes.io/projected/69bfb1df-6b20-422a-9264-bd89506ab8d4-kube-api-access-9hsmz\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.703015 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.702971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.703015 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.702997    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.703274 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-trusted-ca\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.703274 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703252    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7a040f9a-b722-4aa6-9cf3-371f02ee172a-tmp\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703400 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.703374    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.203335249 +0000 UTC m=+35.073378194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : configmap references non-existent config key: service-ca.crt
Sep 11 17:39:11.703400 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.703380    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:11.703504 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.703443    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls podName:921d4454-3d1d-489b-a9d6-31f5e8af7dc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.203426214 +0000 UTC m=+35.073469144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-p6tmp" (UID: "921d4454-3d1d-489b-a9d6-31f5e8af7dc5") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:11.703563 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703538    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-stats-auth\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.703623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703571    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc070bb0-e27e-45c8-b7e5-987dabdbef97-tmp-dir\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.703623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73c2ef90-9bd5-4110-892d-296f12551de4-config\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.703713 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703629    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z7jx\" (UniqueName: \"kubernetes.io/projected/7a040f9a-b722-4aa6-9cf3-371f02ee172a-kube-api-access-4z7jx\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703713 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703658    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:11.703713 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a040f9a-b722-4aa6-9cf3-371f02ee172a-serving-cert\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703712    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.703819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703737    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/041f6f00-15f9-48cf-82a2-fe253ff1f70a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:11.703819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703764    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh57m\" (UniqueName: \"kubernetes.io/projected/a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb-kube-api-access-dh57m\") pod \"volume-data-source-validator-86695b7cc7-nvxnn\" (UID: \"a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"
Sep 11 17:39:11.703819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703792    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7dcv\" (UniqueName: \"kubernetes.io/projected/cd176d7f-b655-4a6a-90d3-7df96b78de54-kube-api-access-n7dcv\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703852    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-installation-pull-secrets\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73c2ef90-9bd5-4110-892d-296f12551de4-serving-cert\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703920    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzvvh\" (UniqueName: \"kubernetes.io/projected/aaae7d8f-4e08-4d25-ad88-0253fb26e743-kube-api-access-gzvvh\") pod \"network-check-source-6f8b768455-2hq8s\" (UID: \"aaae7d8f-4e08-4d25-ad88-0253fb26e743\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703937    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a040f9a-b722-4aa6-9cf3-371f02ee172a-trusted-ca-bundle\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd176d7f-b655-4a6a-90d3-7df96b78de54-config\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.703991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070bb0-e27e-45c8-b7e5-987dabdbef97-config-volume\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704001    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsmhw\" (UniqueName: \"kubernetes.io/projected/26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1-kube-api-access-wsmhw\") pod \"ingress-canary-zzp4k\" (UID: \"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1\") " pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704031    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd176d7f-b655-4a6a-90d3-7df96b78de54-trusted-ca\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704057    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzhlw\" (UniqueName: \"kubernetes.io/projected/79436815-d9f1-4115-a80d-c486e8f70f9c-kube-api-access-zzhlw\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704081    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-certificates\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e471abfe-aaf5-45bb-8f10-9ec40578bf95-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704158    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shwq5\" (UniqueName: \"kubernetes.io/projected/73c2ef90-9bd5-4110-892d-296f12551de4-kube-api-access-shwq5\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704174    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-trusted-ca\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704186    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/73c2ef90-9bd5-4110-892d-296f12551de4-config\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704199    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-image-registry-private-configuration\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.703683    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a040f9a-b722-4aa6-9cf3-371f02ee172a-service-ca-bundle\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704290    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncv24\" (UniqueName: \"kubernetes.io/projected/e471abfe-aaf5-45bb-8f10-9ec40578bf95-kube-api-access-ncv24\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.704314 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd176d7f-b655-4a6a-90d3-7df96b78de54-serving-cert\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704369    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704436    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1-cert\") pod \"ingress-canary-zzp4k\" (UID: \"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1\") " pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.704484    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e471abfe-aaf5-45bb-8f10-9ec40578bf95-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7a040f9a-b722-4aa6-9cf3-371f02ee172a-snapshots\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.704542    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.204527711 +0000 UTC m=+35.074570640 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : secret "router-metrics-certs-default" not found
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hlvh\" (UniqueName: \"kubernetes.io/projected/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-kube-api-access-2hlvh\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704604    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fhrx\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-kube-api-access-9fhrx\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704631    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-bound-sa-token\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-default-certificate\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc2gb\" (UniqueName: \"kubernetes.io/projected/fc070bb0-e27e-45c8-b7e5-987dabdbef97-kube-api-access-jc2gb\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.704961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704708    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-ca-trust-extracted\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.705488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.704965    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cd176d7f-b655-4a6a-90d3-7df96b78de54-trusted-ca\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.705488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.705017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-ca-trust-extracted\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.705488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.705225    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-telemetry-config\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.705488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.705318    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7a040f9a-b722-4aa6-9cf3-371f02ee172a-snapshots\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.705488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.705414    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-certificates\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.705887    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.705903    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bcc4bd74-qz5zs: secret "image-registry-tls" not found
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.705947    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls podName:d70ffa47-c8d2-491b-a8bc-550a503e9b3d nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.20593014 +0000 UTC m=+35.075973075 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls") pod "image-registry-8bcc4bd74-qz5zs" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d") : secret "image-registry-tls" not found
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.706275    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/041f6f00-15f9-48cf-82a2-fe253ff1f70a-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.706384    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.706438    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert podName:041f6f00-15f9-48cf-82a2-fe253ff1f70a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.206423437 +0000 UTC m=+35.076466386 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-qwtvf" (UID: "041f6f00-15f9-48cf-82a2-fe253ff1f70a") : secret "networking-console-plugin-cert" not found
Sep 11 17:39:11.706990 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.706872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/cd176d7f-b655-4a6a-90d3-7df96b78de54-config\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.707412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.707390    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e471abfe-aaf5-45bb-8f10-9ec40578bf95-config\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.709090 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.709057    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e471abfe-aaf5-45bb-8f10-9ec40578bf95-serving-cert\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.709475 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.709451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/73c2ef90-9bd5-4110-892d-296f12551de4-serving-cert\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.709760 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.709707    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/cd176d7f-b655-4a6a-90d3-7df96b78de54-serving-cert\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.710530 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.710419    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-stats-auth\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.710666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.710637    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-installation-pull-secrets\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.711124 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.711098    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7a040f9a-b722-4aa6-9cf3-371f02ee172a-serving-cert\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.711248 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.711224    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-default-certificate\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.712433 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.712154    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-image-registry-private-configuration\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.712827 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.712802    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z7jx\" (UniqueName: \"kubernetes.io/projected/7a040f9a-b722-4aa6-9cf3-371f02ee172a-kube-api-access-4z7jx\") pod \"insights-operator-5844b856db-ht8lt\" (UID: \"7a040f9a-b722-4aa6-9cf3-371f02ee172a\") " pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.713942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.713588    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncv24\" (UniqueName: \"kubernetes.io/projected/e471abfe-aaf5-45bb-8f10-9ec40578bf95-kube-api-access-ncv24\") pod \"kube-storage-version-migrator-operator-6c8f84f897-xls2m\" (UID: \"e471abfe-aaf5-45bb-8f10-9ec40578bf95\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.714870 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.714831    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hsmz\" (UniqueName: \"kubernetes.io/projected/69bfb1df-6b20-422a-9264-bd89506ab8d4-kube-api-access-9hsmz\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:11.715386 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.715316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hlvh\" (UniqueName: \"kubernetes.io/projected/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-kube-api-access-2hlvh\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:11.715706 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.715681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh57m\" (UniqueName: \"kubernetes.io/projected/a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb-kube-api-access-dh57m\") pod \"volume-data-source-validator-86695b7cc7-nvxnn\" (UID: \"a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"
Sep 11 17:39:11.716558 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.716519    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7dcv\" (UniqueName: \"kubernetes.io/projected/cd176d7f-b655-4a6a-90d3-7df96b78de54-kube-api-access-n7dcv\") pod \"console-operator-696884fb9c-bnb6l\" (UID: \"cd176d7f-b655-4a6a-90d3-7df96b78de54\") " pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.716894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.716866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-bound-sa-token\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.717394 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.717371    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fhrx\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-kube-api-access-9fhrx\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:11.717678 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.717658    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzvvh\" (UniqueName: \"kubernetes.io/projected/aaae7d8f-4e08-4d25-ad88-0253fb26e743-kube-api-access-gzvvh\") pod \"network-check-source-6f8b768455-2hq8s\" (UID: \"aaae7d8f-4e08-4d25-ad88-0253fb26e743\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"
Sep 11 17:39:11.718165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.718130    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shwq5\" (UniqueName: \"kubernetes.io/projected/73c2ef90-9bd5-4110-892d-296f12551de4-kube-api-access-shwq5\") pod \"service-ca-operator-cf7c776b7-fqn8k\" (UID: \"73c2ef90-9bd5-4110-892d-296f12551de4\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.740698 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.740674    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:11.740698 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.740690    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:11.740853 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.740675    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:11.743136 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.743115    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 17:39:11.743251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.743116    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 17:39:11.743334 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.743298    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hwqhm\""
Sep 11 17:39:11.743334 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.743325    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7q2fl\""
Sep 11 17:39:11.778101 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.778069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:11.805253 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.805234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.805926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc070bb0-e27e-45c8-b7e5-987dabdbef97-tmp-dir\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.805981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.806078    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070bb0-e27e-45c8-b7e5-987dabdbef97-config-volume\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.806109    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsmhw\" (UniqueName: \"kubernetes.io/projected/26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1-kube-api-access-wsmhw\") pod \"ingress-canary-zzp4k\" (UID: \"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1\") " pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.806144    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zzhlw\" (UniqueName: \"kubernetes.io/projected/79436815-d9f1-4115-a80d-c486e8f70f9c-kube-api-access-zzhlw\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.806220    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.806249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1-cert\") pod \"ingress-canary-zzp4k\" (UID: \"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1\") " pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.806303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc2gb\" (UniqueName: \"kubernetes.io/projected/fc070bb0-e27e-45c8-b7e5-987dabdbef97-kube-api-access-jc2gb\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.807015    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc070bb0-e27e-45c8-b7e5-987dabdbef97-tmp-dir\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.807075    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.807137    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls podName:fc070bb0-e27e-45c8-b7e5-987dabdbef97 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.3071173 +0000 UTC m=+35.177160240 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls") pod "dns-default-545rn" (UID: "fc070bb0-e27e-45c8-b7e5-987dabdbef97") : secret "dns-default-metrics-tls" not found
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.807323    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:11.807405    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls podName:79436815-d9f1-4115-a80d-c486e8f70f9c nodeName:}" failed. No retries permitted until 2026-09-11 17:39:12.307380333 +0000 UTC m=+35.177423274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v67mt" (UID: "79436815-d9f1-4115-a80d-c486e8f70f9c") : secret "samples-operator-tls" not found
Sep 11 17:39:11.809643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.807866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/fc070bb0-e27e-45c8-b7e5-987dabdbef97-config-volume\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.811293 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.811250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1-cert\") pod \"ingress-canary-zzp4k\" (UID: \"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1\") " pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.816792 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.816768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzhlw\" (UniqueName: \"kubernetes.io/projected/79436815-d9f1-4115-a80d-c486e8f70f9c-kube-api-access-zzhlw\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:11.817116 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.817092    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc2gb\" (UniqueName: \"kubernetes.io/projected/fc070bb0-e27e-45c8-b7e5-987dabdbef97-kube-api-access-jc2gb\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:11.817194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.817161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsmhw\" (UniqueName: \"kubernetes.io/projected/26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1-kube-api-access-wsmhw\") pod \"ingress-canary-zzp4k\" (UID: \"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1\") " pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:11.819949 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.819931    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-ht8lt"
Sep 11 17:39:11.828451 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.828429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"
Sep 11 17:39:11.837196 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.837174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:11.864983 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.864742    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"
Sep 11 17:39:11.881359 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.881008    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"
Sep 11 17:39:11.902760 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:11.902456    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zzp4k"
Sep 11 17:39:12.116277 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.115847    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn"]
Sep 11 17:39:12.120166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.119884    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-ht8lt"]
Sep 11 17:39:12.123621 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.123569    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m"]
Sep 11 17:39:12.138379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.138365    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-696884fb9c-bnb6l"]
Sep 11 17:39:12.182111 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.182071    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k"]
Sep 11 17:39:12.205545 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.205385    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s"]
Sep 11 17:39:12.211689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.211530    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:12.211689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.211573    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:12.211689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.211621    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:12.211689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.211650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212380    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212433    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.21240905 +0000 UTC m=+36.082451999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : configmap references non-existent config key: service-ca.crt
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212468    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls podName:921d4454-3d1d-489b-a9d6-31f5e8af7dc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.212449999 +0000 UTC m=+36.082492924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-p6tmp" (UID: "921d4454-3d1d-489b-a9d6-31f5e8af7dc5") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212499    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212513    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bcc4bd74-qz5zs: secret "image-registry-tls" not found
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.212548    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:12.212575 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212556    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls podName:d70ffa47-c8d2-491b-a8bc-550a503e9b3d nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.212537699 +0000 UTC m=+36.082580626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls") pod "image-registry-8bcc4bd74-qz5zs" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d") : secret "image-registry-tls" not found
Sep 11 17:39:12.212987 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.212607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zzp4k"]
Sep 11 17:39:12.212987 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212622    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:39:12.212987 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212678    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.212664718 +0000 UTC m=+36.082707662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : secret "router-metrics-certs-default" not found
Sep 11 17:39:12.212987 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212626    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:39:12.212987 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.212719    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert podName:041f6f00-15f9-48cf-82a2-fe253ff1f70a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.212710382 +0000 UTC m=+36.082753326 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-qwtvf" (UID: "041f6f00-15f9-48cf-82a2-fe253ff1f70a") : secret "networking-console-plugin-cert" not found
Sep 11 17:39:12.313286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.313256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:12.314101 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:12.313401    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:12.314101 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.313439    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:39:12.314101 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.313518    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls podName:79436815-d9f1-4115-a80d-c486e8f70f9c nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.313495655 +0000 UTC m=+36.183538596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v67mt" (UID: "79436815-d9f1-4115-a80d-c486e8f70f9c") : secret "samples-operator-tls" not found
Sep 11 17:39:12.314101 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.313595    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:39:12.314101 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:12.313672    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls podName:fc070bb0-e27e-45c8-b7e5-987dabdbef97 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:13.313653074 +0000 UTC m=+36.183696012 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls") pod "dns-default-545rn" (UID: "fc070bb0-e27e-45c8-b7e5-987dabdbef97") : secret "dns-default-metrics-tls" not found
Sep 11 17:39:13.221183 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.221145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:13.221183 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.221192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:13.221443 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221303    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:39:13.221443 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221307    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:39:13.221443 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221319    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bcc4bd74-qz5zs: secret "image-registry-tls" not found
Sep 11 17:39:13.221443 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.221393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:13.221443 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221425    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls podName:d70ffa47-c8d2-491b-a8bc-550a503e9b3d nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.22140584 +0000 UTC m=+38.091448787 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls") pod "image-registry-8bcc4bd74-qz5zs" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d") : secret "image-registry-tls" not found
Sep 11 17:39:13.221651 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221459    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.221449695 +0000 UTC m=+38.091492619 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : secret "router-metrics-certs-default" not found
Sep 11 17:39:13.221651 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221462    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:39:13.221651 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221509    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert podName:041f6f00-15f9-48cf-82a2-fe253ff1f70a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.221493978 +0000 UTC m=+38.091536916 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-qwtvf" (UID: "041f6f00-15f9-48cf-82a2-fe253ff1f70a") : secret "networking-console-plugin-cert" not found
Sep 11 17:39:13.221651 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.221561    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:13.221651 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.221587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:13.221867 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221667    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:13.221867 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221738    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls podName:921d4454-3d1d-489b-a9d6-31f5e8af7dc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.22170286 +0000 UTC m=+38.091745796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-p6tmp" (UID: "921d4454-3d1d-489b-a9d6-31f5e8af7dc5") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:13.221867 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.221758    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.221749799 +0000 UTC m=+38.091792737 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : configmap references non-existent config key: service-ca.crt
Sep 11 17:39:13.322696 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.322653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:13.323142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:13.322777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:13.323142 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.322801    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:39:13.323142 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.322873    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls podName:fc070bb0-e27e-45c8-b7e5-987dabdbef97 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.322852679 +0000 UTC m=+38.192895622 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls") pod "dns-default-545rn" (UID: "fc070bb0-e27e-45c8-b7e5-987dabdbef97") : secret "dns-default-metrics-tls" not found
Sep 11 17:39:13.323142 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.322931    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:39:13.323142 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:13.322993    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls podName:79436815-d9f1-4115-a80d-c486e8f70f9c nodeName:}" failed. No retries permitted until 2026-09-11 17:39:15.322976874 +0000 UTC m=+38.193019817 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v67mt" (UID: "79436815-d9f1-4115-a80d-c486e8f70f9c") : secret "samples-operator-tls" not found
Sep 11 17:39:13.506630 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.506595    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaae7d8f_4e08_4d25_ad88_0253fb26e743.slice/crio-55b3dddfd450df39aa694db0c58435addb0779397ef2d43530a82bd05e044d9e WatchSource:0}: Error finding container 55b3dddfd450df39aa694db0c58435addb0779397ef2d43530a82bd05e044d9e: Status 404 returned error can't find the container with id 55b3dddfd450df39aa694db0c58435addb0779397ef2d43530a82bd05e044d9e
Sep 11 17:39:13.507242 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.507213    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a040f9a_b722_4aa6_9cf3_371f02ee172a.slice/crio-0037c0a66ad8f8c18981994b21a6a9115ea37b53843071477c199e1a10e52fd9 WatchSource:0}: Error finding container 0037c0a66ad8f8c18981994b21a6a9115ea37b53843071477c199e1a10e52fd9: Status 404 returned error can't find the container with id 0037c0a66ad8f8c18981994b21a6a9115ea37b53843071477c199e1a10e52fd9
Sep 11 17:39:13.508198 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.508061    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8cb7ac8_7ea0_48db_94bc_49f00c7f1eeb.slice/crio-c13771529a33b7d880aec566e3871c9e92293f845c12cb49b66ec99f2fdf9125 WatchSource:0}: Error finding container c13771529a33b7d880aec566e3871c9e92293f845c12cb49b66ec99f2fdf9125: Status 404 returned error can't find the container with id c13771529a33b7d880aec566e3871c9e92293f845c12cb49b66ec99f2fdf9125
Sep 11 17:39:13.509138 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.509118    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd176d7f_b655_4a6a_90d3_7df96b78de54.slice/crio-24be0bc97c76fc30c47423cf0df836c8549eb7c32e1bb943a1b5ac708755003b WatchSource:0}: Error finding container 24be0bc97c76fc30c47423cf0df836c8549eb7c32e1bb943a1b5ac708755003b: Status 404 returned error can't find the container with id 24be0bc97c76fc30c47423cf0df836c8549eb7c32e1bb943a1b5ac708755003b
Sep 11 17:39:13.510443 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.510033    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c2ef90_9bd5_4110_892d_296f12551de4.slice/crio-614329baeed38d451075d72e6369f0551b5da26f12f20f2d2484bb698dda23fe WatchSource:0}: Error finding container 614329baeed38d451075d72e6369f0551b5da26f12f20f2d2484bb698dda23fe: Status 404 returned error can't find the container with id 614329baeed38d451075d72e6369f0551b5da26f12f20f2d2484bb698dda23fe
Sep 11 17:39:13.510889 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.510771    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26fa16d6_41b9_4a5f_b1f9_b7b0c65c63b1.slice/crio-01ace08c26d5508eb66786e83c70a2868e0f2de182a8b9c24b59608694b0f158 WatchSource:0}: Error finding container 01ace08c26d5508eb66786e83c70a2868e0f2de182a8b9c24b59608694b0f158: Status 404 returned error can't find the container with id 01ace08c26d5508eb66786e83c70a2868e0f2de182a8b9c24b59608694b0f158
Sep 11 17:39:13.510997 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:13.510965    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode471abfe_aaf5_45bb_8f10_9ec40578bf95.slice/crio-1fee4a01469480bfd885b60d9565c40c30d24004dd397f68dbceeb5622242f49 WatchSource:0}: Error finding container 1fee4a01469480bfd885b60d9565c40c30d24004dd397f68dbceeb5622242f49: Status 404 returned error can't find the container with id 1fee4a01469480bfd885b60d9565c40c30d24004dd397f68dbceeb5622242f49
Sep 11 17:39:14.042400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.042251    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zzp4k" event={"ID":"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1","Type":"ContainerStarted","Data":"01ace08c26d5508eb66786e83c70a2868e0f2de182a8b9c24b59608694b0f158"}
Sep 11 17:39:14.043920 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.043871    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s" event={"ID":"aaae7d8f-4e08-4d25-ad88-0253fb26e743","Type":"ContainerStarted","Data":"55b3dddfd450df39aa694db0c58435addb0779397ef2d43530a82bd05e044d9e"}
Sep 11 17:39:14.045516 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.045462    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k" event={"ID":"73c2ef90-9bd5-4110-892d-296f12551de4","Type":"ContainerStarted","Data":"614329baeed38d451075d72e6369f0551b5da26f12f20f2d2484bb698dda23fe"}
Sep 11 17:39:14.050085 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.050059    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d27ca5d-8fc8-47a1-b7e0-af23330951d6" containerID="e6990f43d7593fc6a5e471544279208ba1252af33e412fde8636b61617014863" exitCode=0
Sep 11 17:39:14.050181 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.050142    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerDied","Data":"e6990f43d7593fc6a5e471544279208ba1252af33e412fde8636b61617014863"}
Sep 11 17:39:14.053281 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.053247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" event={"ID":"cd176d7f-b655-4a6a-90d3-7df96b78de54","Type":"ContainerStarted","Data":"24be0bc97c76fc30c47423cf0df836c8549eb7c32e1bb943a1b5ac708755003b"}
Sep 11 17:39:14.055868 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.055810    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m" event={"ID":"e471abfe-aaf5-45bb-8f10-9ec40578bf95","Type":"ContainerStarted","Data":"1fee4a01469480bfd885b60d9565c40c30d24004dd397f68dbceeb5622242f49"}
Sep 11 17:39:14.058131 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.058104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ht8lt" event={"ID":"7a040f9a-b722-4aa6-9cf3-371f02ee172a","Type":"ContainerStarted","Data":"0037c0a66ad8f8c18981994b21a6a9115ea37b53843071477c199e1a10e52fd9"}
Sep 11 17:39:14.059865 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:14.059842    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn" event={"ID":"a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb","Type":"ContainerStarted","Data":"c13771529a33b7d880aec566e3871c9e92293f845c12cb49b66ec99f2fdf9125"}
Sep 11 17:39:15.074701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.073722    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d27ca5d-8fc8-47a1-b7e0-af23330951d6" containerID="30663e52c96d38c28518a1afcbd6991faf62d6fa51c13a8e7f71d8f1d277bc2c" exitCode=0
Sep 11 17:39:15.074701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.073802    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerDied","Data":"30663e52c96d38c28518a1afcbd6991faf62d6fa51c13a8e7f71d8f1d277bc2c"}
Sep 11 17:39:15.105405 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.105215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.243501    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.243552    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.243680    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.243777    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.243805    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.243927    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.243986    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls podName:921d4454-3d1d-489b-a9d6-31f5e8af7dc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.243969428 +0000 UTC m=+42.114012368 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-p6tmp" (UID: "921d4454-3d1d-489b-a9d6-31f5e8af7dc5") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244507    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244567    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.244549007 +0000 UTC m=+42.114591975 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : secret "router-metrics-certs-default" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244583    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244635    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert podName:041f6f00-15f9-48cf-82a2-fe253ff1f70a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.244619709 +0000 UTC m=+42.114662634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-qwtvf" (UID: "041f6f00-15f9-48cf-82a2-fe253ff1f70a") : secret "networking-console-plugin-cert" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244635    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244653    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bcc4bd74-qz5zs: secret "image-registry-tls" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244707    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls podName:d70ffa47-c8d2-491b-a8bc-550a503e9b3d nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.244695133 +0000 UTC m=+42.114738083 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls") pod "image-registry-8bcc4bd74-qz5zs" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d") : secret "image-registry-tls" not found
Sep 11 17:39:15.245033 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.244936    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.244923133 +0000 UTC m=+42.114966078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : configmap references non-existent config key: service-ca.crt
Sep 11 17:39:15.248537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.248431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:15.345497 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.344658    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:15.345497 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.344771    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:15.345497 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.344956    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:39:15.345497 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.345016    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls podName:79436815-d9f1-4115-a80d-c486e8f70f9c nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.344998932 +0000 UTC m=+42.215041874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v67mt" (UID: "79436815-d9f1-4115-a80d-c486e8f70f9c") : secret "samples-operator-tls" not found
Sep 11 17:39:15.345497 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.345378    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:39:15.345497 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:15.345423    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls podName:fc070bb0-e27e-45c8-b7e5-987dabdbef97 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:19.345407941 +0000 UTC m=+42.215450877 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls") pod "dns-default-545rn" (UID: "fc070bb0-e27e-45c8-b7e5-987dabdbef97") : secret "dns-default-metrics-tls" not found
Sep 11 17:39:15.347655 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:15.347590    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:16.059056 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.058974    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:16.083793 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.083712    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" event={"ID":"9d27ca5d-8fc8-47a1-b7e0-af23330951d6","Type":"ContainerStarted","Data":"450e132d2db6cfa1a782f732f07e8ce5fb04ae5ec1a3a0cd4cd96bee13668e92"}
Sep 11 17:39:16.105535 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.105473    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pmgpd" podStartSLOduration=7.999286247 podStartE2EDuration="38.10545046s" totalImagesPullingTime="30.106164213s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:16.103681354 +0000 UTC m=+38.973724300" watchObservedRunningTime="2026-09-11 17:39:16.10545046 +0000 UTC m=+38.975493406"
Sep 11 17:39:16.181727 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.181202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:16.283061 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.283029    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:16.455130 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.455037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:16.520334 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:16.520306    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:17.280305 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:17.280272    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:17.380212 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:17.380179    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:17.660329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:17.660234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:17.738493 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:17.738423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:18.957816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:18.957767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:19.283790 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.283757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:19.283790 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.283794    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.283837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.283869    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.283939    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.283953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.283987    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.283963669 +0000 UTC m=+50.154006606 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : configmap references non-existent config key: service-ca.crt
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284004    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284016    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284034    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls podName:921d4454-3d1d-489b-a9d6-31f5e8af7dc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.284023146 +0000 UTC m=+50.154066073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-p6tmp" (UID: "921d4454-3d1d-489b-a9d6-31f5e8af7dc5") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284038    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284049    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bcc4bd74-qz5zs: secret "image-registry-tls" not found
Sep 11 17:39:19.284049 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284057    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert podName:041f6f00-15f9-48cf-82a2-fe253ff1f70a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.284040598 +0000 UTC m=+50.154083541 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-qwtvf" (UID: "041f6f00-15f9-48cf-82a2-fe253ff1f70a") : secret "networking-console-plugin-cert" not found
Sep 11 17:39:19.284442 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284084    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls podName:d70ffa47-c8d2-491b-a8bc-550a503e9b3d nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.284070665 +0000 UTC m=+50.154113599 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls") pod "image-registry-8bcc4bd74-qz5zs" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d") : secret "image-registry-tls" not found
Sep 11 17:39:19.284442 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.284110    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.284101972 +0000 UTC m=+50.154144900 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : secret "router-metrics-certs-default" not found
Sep 11 17:39:19.385142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.385096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:19.385315 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:19.385242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:19.385315 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.385248    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:39:19.385441 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.385322    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls podName:79436815-d9f1-4115-a80d-c486e8f70f9c nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.385300881 +0000 UTC m=+50.255343810 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v67mt" (UID: "79436815-d9f1-4115-a80d-c486e8f70f9c") : secret "samples-operator-tls" not found
Sep 11 17:39:19.385441 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.385376    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:39:19.385441 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:19.385428    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls podName:fc070bb0-e27e-45c8-b7e5-987dabdbef97 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:27.385412273 +0000 UTC m=+50.255455196 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls") pod "dns-default-545rn" (UID: "fc070bb0-e27e-45c8-b7e5-987dabdbef97") : secret "dns-default-metrics-tls" not found
Sep 11 17:39:21.998207 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:21.998128    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:22.101601 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.101570    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/0.log"
Sep 11 17:39:22.101776 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.101617    2582 generic.go:356] "Generic (PLEG): container finished" podID="cd176d7f-b655-4a6a-90d3-7df96b78de54" containerID="7ab7971c63505db0f952a068582d1be71ada0c5380acd3b98d29c9af06349298" exitCode=255
Sep 11 17:39:22.101776 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.101696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" event={"ID":"cd176d7f-b655-4a6a-90d3-7df96b78de54","Type":"ContainerDied","Data":"7ab7971c63505db0f952a068582d1be71ada0c5380acd3b98d29c9af06349298"}
Sep 11 17:39:22.102085 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.101961    2582 scope.go:117] "RemoveContainer" containerID="7ab7971c63505db0f952a068582d1be71ada0c5380acd3b98d29c9af06349298"
Sep 11 17:39:22.104103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.104055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m" event={"ID":"e471abfe-aaf5-45bb-8f10-9ec40578bf95","Type":"ContainerStarted","Data":"6c5f1ebbd76638ccd574b891a9b5670d4afd2c411ea40a29e6237be6de4411c3"}
Sep 11 17:39:22.105865 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.105840    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ht8lt" event={"ID":"7a040f9a-b722-4aa6-9cf3-371f02ee172a","Type":"ContainerStarted","Data":"19868478c9547187fd03b892a7dad0eec8479c9f9ba6ea622a1263823f765615"}
Sep 11 17:39:22.107465 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.107441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn" event={"ID":"a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb","Type":"ContainerStarted","Data":"b0d5f4e0c3946cd1cd3b74aaa9187beca1402fa33c0c1ba9e1d418d5abc6946e"}
Sep 11 17:39:22.110657 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.109537    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zzp4k" event={"ID":"26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1","Type":"ContainerStarted","Data":"1bf062951db2a035c88439823e91a670b7ff3d0782ebb181f741c1206871ef2d"}
Sep 11 17:39:22.112274 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.112252    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s" event={"ID":"aaae7d8f-4e08-4d25-ad88-0253fb26e743","Type":"ContainerStarted","Data":"6957fbc5be886b19bd1894d66d4fd851c6088181cbb043b292a9ba940222fae8"}
Sep 11 17:39:22.114123 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.114101    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k" event={"ID":"73c2ef90-9bd5-4110-892d-296f12551de4","Type":"ContainerStarted","Data":"7a7e501a024976fe0620628adeb56064b68c9fe595275c49911538b4ad090e65"}
Sep 11 17:39:22.139776 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.139727    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-ht8lt" podStartSLOduration=7.2889189 podStartE2EDuration="15.139708163s" totalImagesPullingTime="7.850789263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:22.138661715 +0000 UTC m=+45.008704661" watchObservedRunningTime="2026-09-11 17:39:22.139708163 +0000 UTC m=+45.009751128"
Sep 11 17:39:22.155086 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.155039    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k" podStartSLOduration=7.307578203 podStartE2EDuration="15.155020888s" totalImagesPullingTime="7.847442685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:22.154616034 +0000 UTC m=+45.024658981" watchObservedRunningTime="2026-09-11 17:39:22.155020888 +0000 UTC m=+45.025063836"
Sep 11 17:39:22.189808 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.189753    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m" podStartSLOduration=7.344164099 podStartE2EDuration="15.189732415s" totalImagesPullingTime="7.845568316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:22.188406317 +0000 UTC m=+45.058449260" watchObservedRunningTime="2026-09-11 17:39:22.189732415 +0000 UTC m=+45.059775358"
Sep 11 17:39:22.189985 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.189961    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-86695b7cc7-nvxnn" podStartSLOduration=7.705800557 podStartE2EDuration="15.189952492s" totalImagesPullingTime="7.484151935s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:22.171597083 +0000 UTC m=+45.041640028" watchObservedRunningTime="2026-09-11 17:39:22.189952492 +0000 UTC m=+45.059995433"
Sep 11 17:39:22.207920 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.207662    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-2hq8s" podStartSLOduration=7.347280057 podStartE2EDuration="15.207641407s" totalImagesPullingTime="7.86036135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:22.205363306 +0000 UTC m=+45.075406244" watchObservedRunningTime="2026-09-11 17:39:22.207641407 +0000 UTC m=+45.077684353"
Sep 11 17:39:22.293762 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.293729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:22.394394 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.394309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:22.782738 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.782702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:22.885408 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.885338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:22.949555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.949503    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zzp4k" podStartSLOduration=4.103216337 podStartE2EDuration="11.949481361s" totalImagesPullingTime="7.846265024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:22.222331554 +0000 UTC m=+45.092374500" watchObservedRunningTime="2026-09-11 17:39:22.949481361 +0000 UTC m=+45.819524305"
Sep 11 17:39:22.950185 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.950167    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zw2xl"]
Sep 11 17:39:22.972541 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.972515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:22.983256 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.983224    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zw2xl"]
Sep 11 17:39:22.984629 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.984611    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 17:39:22.984722 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.984613    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 17:39:22.984722 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:22.984657    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-72jzj\""
Sep 11 17:39:23.012342 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.012317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.018614 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.018591    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-crio-socket\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.018726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.018632    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-data-volume\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.018726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.018663    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.018726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.018692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.018726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.018718    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hnt4\" (UniqueName: \"kubernetes.io/projected/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-api-access-6hnt4\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119204    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-crio-socket\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119231    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:39:23.119289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-data-volume\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119368    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119410    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hnt4\" (UniqueName: \"kubernetes.io/projected/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-api-access-6hnt4\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119555 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.119427    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:39:23.119555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119474    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-crio-socket\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119555 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.119481    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:23.619463343 +0000 UTC m=+46.489506270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : configmap "kube-rbac-proxy" not found
Sep 11 17:39:23.119832 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.119561    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:23.119832 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119592    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-data-volume\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.119832 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.119619    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:23.619601547 +0000 UTC m=+46.489644494 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:23.119832 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119679    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/0.log"
Sep 11 17:39:23.119832 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119706    2582 generic.go:356] "Generic (PLEG): container finished" podID="cd176d7f-b655-4a6a-90d3-7df96b78de54" containerID="d7fd4fa2bf0cffc482003a0689f559722a71b368462ff85cfc8d6943ee3ff26c" exitCode=255
Sep 11 17:39:23.119832 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119791    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" event={"ID":"cd176d7f-b655-4a6a-90d3-7df96b78de54","Type":"ContainerDied","Data":"d7fd4fa2bf0cffc482003a0689f559722a71b368462ff85cfc8d6943ee3ff26c"}
Sep 11 17:39:23.120078 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.119837    2582 scope.go:117] "RemoveContainer" containerID="7ab7971c63505db0f952a068582d1be71ada0c5380acd3b98d29c9af06349298"
Sep 11 17:39:23.120078 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.120037    2582 scope.go:117] "RemoveContainer" containerID="d7fd4fa2bf0cffc482003a0689f559722a71b368462ff85cfc8d6943ee3ff26c"
Sep 11 17:39:23.120240 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.120216    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bnb6l_openshift-console-operator(cd176d7f-b655-4a6a-90d3-7df96b78de54)\"" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" podUID="cd176d7f-b655-4a6a-90d3-7df96b78de54"
Sep 11 17:39:23.138343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.138304    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hnt4\" (UniqueName: \"kubernetes.io/projected/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-api-access-6hnt4\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.209087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.209054    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"]
Sep 11 17:39:23.214761 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.214715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.250813 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.250788    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"]
Sep 11 17:39:23.250909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.250890    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"
Sep 11 17:39:23.253082 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.253061    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 17:39:23.253296 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.253280    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-hrfpb\""
Sep 11 17:39:23.253461 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.253447    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 17:39:23.286343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.286317    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.322667 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.322642    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbb68\" (UniqueName: \"kubernetes.io/projected/cc581405-de86-4e56-bb95-d8f7edbf1e81-kube-api-access-xbb68\") pod \"migrator-8b4d697c5-5mkfm\" (UID: \"cc581405-de86-4e56-bb95-d8f7edbf1e81\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"
Sep 11 17:39:23.353872 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.353849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.424189 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.424121    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbb68\" (UniqueName: \"kubernetes.io/projected/cc581405-de86-4e56-bb95-d8f7edbf1e81-kube-api-access-xbb68\") pod \"migrator-8b4d697c5-5mkfm\" (UID: \"cc581405-de86-4e56-bb95-d8f7edbf1e81\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"
Sep 11 17:39:23.432257 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.432239    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbb68\" (UniqueName: \"kubernetes.io/projected/cc581405-de86-4e56-bb95-d8f7edbf1e81-kube-api-access-xbb68\") pod \"migrator-8b4d697c5-5mkfm\" (UID: \"cc581405-de86-4e56-bb95-d8f7edbf1e81\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"
Sep 11 17:39:23.499151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.499116    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.559567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.559539    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"
Sep 11 17:39:23.600246 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.600218    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:23.626380 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.626335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.626503 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.626405    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:23.626503 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.626431    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:39:23.626603 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.626521    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:24.626498679 +0000 UTC m=+47.496541620 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : configmap "kube-rbac-proxy" not found
Sep 11 17:39:23.626603 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.626558    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:23.626679 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:23.626606    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:24.626593204 +0000 UTC m=+47.496636142 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:23.851545 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:23.851507    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm"]
Sep 11 17:39:23.854209 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:23.854176    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc581405_de86_4e56_bb95_d8f7edbf1e81.slice/crio-d8cd8e4b0be294f4f8caa40708fc2c41e3ea9029b84b6dcd9a9ad70eac30cfec WatchSource:0}: Error finding container d8cd8e4b0be294f4f8caa40708fc2c41e3ea9029b84b6dcd9a9ad70eac30cfec: Status 404 returned error can't find the container with id d8cd8e4b0be294f4f8caa40708fc2c41e3ea9029b84b6dcd9a9ad70eac30cfec
Sep 11 17:39:24.125161 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:24.125052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm" event={"ID":"cc581405-de86-4e56-bb95-d8f7edbf1e81","Type":"ContainerStarted","Data":"d8cd8e4b0be294f4f8caa40708fc2c41e3ea9029b84b6dcd9a9ad70eac30cfec"}
Sep 11 17:39:24.126747 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:24.126728    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:39:24.127133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:24.127116    2582 scope.go:117] "RemoveContainer" containerID="d7fd4fa2bf0cffc482003a0689f559722a71b368462ff85cfc8d6943ee3ff26c"
Sep 11 17:39:24.127326 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:24.127306    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bnb6l_openshift-console-operator(cd176d7f-b655-4a6a-90d3-7df96b78de54)\"" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" podUID="cd176d7f-b655-4a6a-90d3-7df96b78de54"
Sep 11 17:39:24.310760 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:24.310732    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vv7sb_783d55b1-3f78-4435-9d56-cb60f6f7c2f3/dns-node-resolver/0.log"
Sep 11 17:39:24.633921 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:24.633891    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:24.634075 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:24.633935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:24.634075 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:24.634016    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:39:24.634075 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:24.634034    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:24.634167 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:24.634081    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:26.634065766 +0000 UTC m=+49.504108703 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : configmap "kube-rbac-proxy" not found
Sep 11 17:39:24.634167 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:24.634096    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:26.634090455 +0000 UTC m=+49.504133378 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:25.091400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.091368    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zhgz2"]
Sep 11 17:39:25.124544 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.124508    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zhgz2"]
Sep 11 17:39:25.124693 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.124636    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.126960 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.126935    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 17:39:25.127330 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.127083    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 17:39:25.127582 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.127563    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jdz76\""
Sep 11 17:39:25.127679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.127585    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 17:39:25.127679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.127603    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 17:39:25.134488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.134463    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.139074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.139052    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmqz6\" (UniqueName: \"kubernetes.io/projected/c641252b-393d-4ef8-9670-38298026a4d6-kube-api-access-kmqz6\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.139185 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.139109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c641252b-393d-4ef8-9670-38298026a4d6-signing-cabundle\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.139185 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.139176    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c641252b-393d-4ef8-9670-38298026a4d6-signing-key\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.196970 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.196930    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.239988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.239961    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmqz6\" (UniqueName: \"kubernetes.io/projected/c641252b-393d-4ef8-9670-38298026a4d6-kube-api-access-kmqz6\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.240149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.240006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c641252b-393d-4ef8-9670-38298026a4d6-signing-cabundle\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.240149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.240036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c641252b-393d-4ef8-9670-38298026a4d6-signing-key\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.240879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.240859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c641252b-393d-4ef8-9670-38298026a4d6-signing-cabundle\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.242504 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.242486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c641252b-393d-4ef8-9670-38298026a4d6-signing-key\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.248405 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.248380    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmqz6\" (UniqueName: \"kubernetes.io/projected/c641252b-393d-4ef8-9670-38298026a4d6-kube-api-access-kmqz6\") pod \"service-ca-568869fdc5-zhgz2\" (UID: \"c641252b-393d-4ef8-9670-38298026a4d6\") " pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.310695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.310671    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xx9jz_f6fe38a1-c4b8-4cdd-8156-27a378e522f7/node-ca/0.log"
Sep 11 17:39:25.436468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.436407    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-zhgz2"
Sep 11 17:39:25.740322 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.740290    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zzp4k_26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1/serve-healthcheck-canary/0.log"
Sep 11 17:39:25.752772 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.752746    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-zhgz2"]
Sep 11 17:39:25.755843 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:25.755819    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc641252b_393d_4ef8_9670_38298026a4d6.slice/crio-7f5cafad1be307c95b7e427ba7c4aee056069b3f5f658c08e1465a245944acf7 WatchSource:0}: Error finding container 7f5cafad1be307c95b7e427ba7c4aee056069b3f5f658c08e1465a245944acf7: Status 404 returned error can't find the container with id 7f5cafad1be307c95b7e427ba7c4aee056069b3f5f658c08e1465a245944acf7
Sep 11 17:39:25.867171 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.867003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:25.967510 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:25.967456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:26.136016 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.135984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-zhgz2" event={"ID":"c641252b-393d-4ef8-9670-38298026a4d6","Type":"ContainerStarted","Data":"19df7a62d12908b6033723217102bd5f462de53d2ecd9474b7d36337384079a6"}
Sep 11 17:39:26.136463 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.136077    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-zhgz2" event={"ID":"c641252b-393d-4ef8-9670-38298026a4d6","Type":"ContainerStarted","Data":"7f5cafad1be307c95b7e427ba7c4aee056069b3f5f658c08e1465a245944acf7"}
Sep 11 17:39:26.137940 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.137916    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm" event={"ID":"cc581405-de86-4e56-bb95-d8f7edbf1e81","Type":"ContainerStarted","Data":"f1bd9971fd400c5312bf816dc8b55dea33e68b1f8e6fbc9a2e3fd773cf203175"}
Sep 11 17:39:26.138044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.137955    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm" event={"ID":"cc581405-de86-4e56-bb95-d8f7edbf1e81","Type":"ContainerStarted","Data":"fa2bebac6e12a2c939fdae1c58c02bb752df26a05b90609df87c31113dad4538"}
Sep 11 17:39:26.152865 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.152826    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-zhgz2" podStartSLOduration=1.15280971 podStartE2EDuration="1.15280971s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:26.151666646 +0000 UTC m=+49.021709593" watchObservedRunningTime="2026-09-11 17:39:26.15280971 +0000 UTC m=+49.022852689"
Sep 11 17:39:26.173266 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.173223    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-5mkfm" podStartSLOduration=1.342228456 podStartE2EDuration="3.173209559s" totalImagesPullingTime="1.830981103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:26.171779388 +0000 UTC m=+49.041822332" watchObservedRunningTime="2026-09-11 17:39:26.173209559 +0000 UTC m=+49.043252563"
Sep 11 17:39:26.219517 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.219454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:26.652428 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.652393    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:26.652634 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:26.652457    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:26.652634 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:26.652535    2582 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 17:39:26.652634 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:26.652559    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:26.652634 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:26.652614    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:30.652596497 +0000 UTC m=+53.522639439 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : configmap "kube-rbac-proxy" not found
Sep 11 17:39:26.652786 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:26.652671    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:30.652657785 +0000 UTC m=+53.522700752 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:27.359505 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.359463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:27.360561 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.360109    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.360089075 +0000 UTC m=+66.230132004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : configmap references non-existent config key: service-ca.crt
Sep 11 17:39:27.360764 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.360736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:27.360907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.360801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:27.360907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.360829    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:27.361020 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.360918    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:27.361121 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361106    2582 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 11 17:39:27.361184 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361154    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs podName:69bfb1df-6b20-422a-9264-bd89506ab8d4 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.361140302 +0000 UTC m=+66.231183231 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs") pod "router-default-89b8b6fbd-68pzd" (UID: "69bfb1df-6b20-422a-9264-bd89506ab8d4") : secret "router-metrics-certs-default" not found
Sep 11 17:39:27.361241 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361206    2582 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:27.361241 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361235    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls podName:921d4454-3d1d-489b-a9d6-31f5e8af7dc5 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.361226031 +0000 UTC m=+66.231268968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cd954b897-p6tmp" (UID: "921d4454-3d1d-489b-a9d6-31f5e8af7dc5") : secret "cluster-monitoring-operator-tls" not found
Sep 11 17:39:27.361342 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361296    2582 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 11 17:39:27.361342 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361306    2582 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-8bcc4bd74-qz5zs: secret "image-registry-tls" not found
Sep 11 17:39:27.361342 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361333    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls podName:d70ffa47-c8d2-491b-a8bc-550a503e9b3d nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.361324573 +0000 UTC m=+66.231367501 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls") pod "image-registry-8bcc4bd74-qz5zs" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d") : secret "image-registry-tls" not found
Sep 11 17:39:27.361522 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361394    2582 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 11 17:39:27.361522 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.361429    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert podName:041f6f00-15f9-48cf-82a2-fe253ff1f70a nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.361419297 +0000 UTC m=+66.231462226 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert") pod "networking-console-plugin-5dcbc7df6b-qwtvf" (UID: "041f6f00-15f9-48cf-82a2-fe253ff1f70a") : secret "networking-console-plugin-cert" not found
Sep 11 17:39:27.388286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.388256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:27.462053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.462014    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:27.462224 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.462151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:27.462333 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.462317    2582 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 11 17:39:27.462424 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.462405    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls podName:79436815-d9f1-4115-a80d-c486e8f70f9c nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.462386451 +0000 UTC m=+66.332429389 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-v67mt" (UID: "79436815-d9f1-4115-a80d-c486e8f70f9c") : secret "samples-operator-tls" not found
Sep 11 17:39:27.462610 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.462591    2582 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 17:39:27.462730 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:27.462720    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls podName:fc070bb0-e27e-45c8-b7e5-987dabdbef97 nodeName:}" failed. No retries permitted until 2026-09-11 17:39:43.462703628 +0000 UTC m=+66.332746569 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls") pod "dns-default-545rn" (UID: "fc070bb0-e27e-45c8-b7e5-987dabdbef97") : secret "dns-default-metrics-tls" not found
Sep 11 17:39:27.634270 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.634167    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:27.736184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.736151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:27.833674 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.833640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:27.932178 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:27.932095    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:28.014197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:28.014159    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:28.128334 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:28.128297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:29.292102 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:29.292067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:29.491228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:29.491189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:29.552232 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:29.552163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:29.680783 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:29.680749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:29.911009 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:29.910920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:30.085828 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:30.085788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:30.153840 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:30.153808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:30.691285 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:30.691253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:30.691686 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:30.691297    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:30.691686 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:30.691430    2582 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:30.691686 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:30.691486    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls podName:d44b26f1-116a-4b7c-8a5c-15a113efe9fd nodeName:}" failed. No retries permitted until 2026-09-11 17:39:38.691470815 +0000 UTC m=+61.561513750 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zw2xl" (UID: "d44b26f1-116a-4b7c-8a5c-15a113efe9fd") : secret "insights-runtime-extractor-tls" not found
Sep 11 17:39:30.691822 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:30.691803    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:30.884201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:30.884159    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:31.073554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.073524    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:31.284011 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.283956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:31.588570 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.588542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:31.688950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.688915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:31.837998 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.837968    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:31.838395 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.838018    2582 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:31.838496 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.838479    2582 scope.go:117] "RemoveContainer" containerID="d7fd4fa2bf0cffc482003a0689f559722a71b368462ff85cfc8d6943ee3ff26c"
Sep 11 17:39:31.838726 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:39:31.838678    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-696884fb9c-bnb6l_openshift-console-operator(cd176d7f-b655-4a6a-90d3-7df96b78de54)\"" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" podUID="cd176d7f-b655-4a6a-90d3-7df96b78de54"
Sep 11 17:39:31.880938 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:31.880906    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:32.195842 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:32.195765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:32.285857 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:32.285821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:32.361721 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:32.361680    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:32.529755 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:32.529721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:32.605535 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:32.605492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:32.904628 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:32.904550    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:33.005151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:33.005118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:33.483294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:33.483267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:33.682301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:33.682259    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:34.074554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:34.074510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:34.277036 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:34.277003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:34.645729 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:34.645696    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:34.746468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:34.746436    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:35.294880 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:35.294846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:35.679886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:35.679804    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.080436 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:36.080400    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.287319 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:36.287278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:36.883255 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:36.883216    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:37.035948 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:37.035922    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nwqjv"
Sep 11 17:39:37.273510 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:37.273475    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:37.481260 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:37.481226    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:37.878966 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:37.878935    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:38.077431 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:38.077394    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:38.381906 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:38.381867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:38.758550 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:38.758520    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:38.760849 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:38.760822    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d44b26f1-116a-4b7c-8a5c-15a113efe9fd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zw2xl\" (UID: \"d44b26f1-116a-4b7c-8a5c-15a113efe9fd\") " pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:38.883433 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:38.883407    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-72jzj\""
Sep 11 17:39:38.892007 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:38.891985    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zw2xl"
Sep 11 17:39:39.175003 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:39.174924    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zw2xl"]
Sep 11 17:39:39.177721 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:39.177690    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd44b26f1_116a_4b7c_8a5c_15a113efe9fd.slice/crio-04a5dc7f34e0e4a871ef34264712de6510cce103ffffbc30f2e23527353117dc WatchSource:0}: Error finding container 04a5dc7f34e0e4a871ef34264712de6510cce103ffffbc30f2e23527353117dc: Status 404 returned error can't find the container with id 04a5dc7f34e0e4a871ef34264712de6510cce103ffffbc30f2e23527353117dc
Sep 11 17:39:40.185286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:40.185205    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zw2xl" event={"ID":"d44b26f1-116a-4b7c-8a5c-15a113efe9fd","Type":"ContainerStarted","Data":"bc92e899d4b6b331dbb4f156ebadcdc411a4491f8a458db0349cafab524dcff7"}
Sep 11 17:39:40.185286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:40.185257    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zw2xl" event={"ID":"d44b26f1-116a-4b7c-8a5c-15a113efe9fd","Type":"ContainerStarted","Data":"745b48843978fc2bc840d86f86f3dd2fddd9ab480ddb6d4c605bc0c5110c7ea1"}
Sep 11 17:39:40.185286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:40.185278    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zw2xl" event={"ID":"d44b26f1-116a-4b7c-8a5c-15a113efe9fd","Type":"ContainerStarted","Data":"04a5dc7f34e0e4a871ef34264712de6510cce103ffffbc30f2e23527353117dc"}
Sep 11 17:39:40.734729 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:40.734691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:40.794658 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:40.794628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:42.198175 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:42.198140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zw2xl" event={"ID":"d44b26f1-116a-4b7c-8a5c-15a113efe9fd","Type":"ContainerStarted","Data":"98d54dc89f0d8718a5260637f862c50f9aeb665acf4a315a185acafaca5ff11c"}
Sep 11 17:39:42.218129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:42.218087    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zw2xl" podStartSLOduration=18.013485274 podStartE2EDuration="20.218074188s" totalImagesPullingTime="2.204588914s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:39:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:42.217875335 +0000 UTC m=+65.087918279" watchObservedRunningTime="2026-09-11 17:39:42.218074188 +0000 UTC m=+65.088117145"
Sep 11 17:39:42.740488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:42.740457    2582 scope.go:117] "RemoveContainer" containerID="d7fd4fa2bf0cffc482003a0689f559722a71b368462ff85cfc8d6943ee3ff26c"
Sep 11 17:39:43.204636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.204564    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:39:43.204996 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.204684    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" event={"ID":"cd176d7f-b655-4a6a-90d3-7df96b78de54","Type":"ContainerStarted","Data":"8ae5bde9b8d23ba3e040b6bf86685471cde66da2327255e605ae35366a4c1429"}
Sep 11 17:39:43.205126 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.205107    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:43.209701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.209674    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l"
Sep 11 17:39:43.225632 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.225241    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-696884fb9c-bnb6l" podStartSLOduration=28.376989878 podStartE2EDuration="36.225221487s" totalImagesPullingTime="7.848231609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:43.224855185 +0000 UTC m=+66.094898130" watchObservedRunningTime="2026-09-11 17:39:43.225221487 +0000 UTC m=+66.095264429"
Sep 11 17:39:43.397427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:43.397427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397428    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:43.397666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397461    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:43.397666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397479    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:43.397666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:43.397666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397534    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:43.397866 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.397692    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:43.398182 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.398155    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69bfb1df-6b20-422a-9264-bd89506ab8d4-service-ca-bundle\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:43.399964 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.399937    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 17:39:43.400069 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.400044    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/921d4454-3d1d-489b-a9d6-31f5e8af7dc5-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cd954b897-p6tmp\" (UID: \"921d4454-3d1d-489b-a9d6-31f5e8af7dc5\") " pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:43.400125 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.400080    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/69bfb1df-6b20-422a-9264-bd89506ab8d4-metrics-certs\") pod \"router-default-89b8b6fbd-68pzd\" (UID: \"69bfb1df-6b20-422a-9264-bd89506ab8d4\") " pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:43.400272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.400255    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 17:39:43.400398 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.400379    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"image-registry-8bcc4bd74-qz5zs\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") " pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:43.400464 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.400431    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/041f6f00-15f9-48cf-82a2-fe253ff1f70a-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-qwtvf\" (UID: \"041f6f00-15f9-48cf-82a2-fe253ff1f70a\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:43.403056 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.403019    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.409715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.409691    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/197940f3-4a76-4c8f-bf3f-c4adcf446604-metrics-certs\") pod \"network-metrics-daemon-7h22x\" (UID: \"197940f3-4a76-4c8f-bf3f-c4adcf446604\") " pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:43.409817 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.409800    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59f9e5b9-528f-4400-a787-3c339153c3b4-original-pull-secret\") pod \"global-pull-secret-syncer-cgzbn\" (UID: \"59f9e5b9-528f-4400-a787-3c339153c3b4\") " pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:43.498553 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.498525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:43.498687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.498581    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:43.498687 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.498645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:43.500736 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.500704    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc070bb0-e27e-45c8-b7e5-987dabdbef97-metrics-tls\") pod \"dns-default-545rn\" (UID: \"fc070bb0-e27e-45c8-b7e5-987dabdbef97\") " pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:43.500858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.500810    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/79436815-d9f1-4115-a80d-c486e8f70f9c-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-v67mt\" (UID: \"79436815-d9f1-4115-a80d-c486e8f70f9c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:43.500954 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.500935    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc44s\" (UniqueName: \"kubernetes.io/projected/c9347efe-d030-4e8f-b60f-9929b060c87a-kube-api-access-rc44s\") pod \"network-check-target-rr858\" (UID: \"c9347efe-d030-4e8f-b60f-9929b060c87a\") " pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:43.502575 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.502550    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.555774 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.555755    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hwqhm\""
Sep 11 17:39:43.561530 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.561516    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-cgzbn"
Sep 11 17:39:43.563572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.563531    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.563663 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.563637    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:43.568572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.568555    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7q2fl\""
Sep 11 17:39:43.577167 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.577149    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-7h22x"
Sep 11 17:39:43.591062 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.591032    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2cprr\""
Sep 11 17:39:43.598942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.598914    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:43.648830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.648806    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-kwmgn\""
Sep 11 17:39:43.658368 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.657176    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:43.658368 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.657512    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-249wm\""
Sep 11 17:39:43.664482 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.664463    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"
Sep 11 17:39:43.675259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.675222    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-bgbb4\""
Sep 11 17:39:43.683811 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.683787    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"
Sep 11 17:39:43.690174 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.689962    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-rxpjs\""
Sep 11 17:39:43.697387 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.697366    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-zb994\""
Sep 11 17:39:43.697619 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.697604    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"
Sep 11 17:39:43.704983 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.704773    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:43.707560 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.706223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.842053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.842031    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rr858"]
Sep 11 17:39:43.843462 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.843436    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-cgzbn"]
Sep 11 17:39:43.844023 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.843997    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f9e5b9_528f_4400_a787_3c339153c3b4.slice/crio-0aa7ae607cf31f4fdd03c437ca6f35c523b4f1bf89991ebb50a2b1bdb53e1d0f WatchSource:0}: Error finding container 0aa7ae607cf31f4fdd03c437ca6f35c523b4f1bf89991ebb50a2b1bdb53e1d0f: Status 404 returned error can't find the container with id 0aa7ae607cf31f4fdd03c437ca6f35c523b4f1bf89991ebb50a2b1bdb53e1d0f
Sep 11 17:39:43.844826 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.844802    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9347efe_d030_4e8f_b60f_9929b060c87a.slice/crio-8b2cd93f246272dd2cc53420d8f9088f8d74ba736b78581247557d9a0e38ca3a WatchSource:0}: Error finding container 8b2cd93f246272dd2cc53420d8f9088f8d74ba736b78581247557d9a0e38ca3a: Status 404 returned error can't find the container with id 8b2cd93f246272dd2cc53420d8f9088f8d74ba736b78581247557d9a0e38ca3a
Sep 11 17:39:43.884787 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.884761    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-7h22x"]
Sep 11 17:39:43.886250 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.886225    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197940f3_4a76_4c8f_bf3f_c4adcf446604.slice/crio-e2db1ad6e979b5f287a9697d1a62f2386de4352aface6a9148410c13eb91f625 WatchSource:0}: Error finding container e2db1ad6e979b5f287a9697d1a62f2386de4352aface6a9148410c13eb91f625: Status 404 returned error can't find the container with id e2db1ad6e979b5f287a9697d1a62f2386de4352aface6a9148410c13eb91f625
Sep 11 17:39:43.904152 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.904125    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8bcc4bd74-qz5zs"]
Sep 11 17:39:43.904288 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.904271    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70ffa47_c8d2_491b_a8bc_550a503e9b3d.slice/crio-90e2d7cbf758583e7c3dd979f74965a8a8b337de598ee53d8d929448b780cac3 WatchSource:0}: Error finding container 90e2d7cbf758583e7c3dd979f74965a8a8b337de598ee53d8d929448b780cac3: Status 404 returned error can't find the container with id 90e2d7cbf758583e7c3dd979f74965a8a8b337de598ee53d8d929448b780cac3
Sep 11 17:39:43.907590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.907569    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:43.969783 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.969757    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-89b8b6fbd-68pzd"]
Sep 11 17:39:43.970681 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.970659    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf"]
Sep 11 17:39:43.972970 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.972948    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041f6f00_15f9_48cf_82a2_fe253ff1f70a.slice/crio-475bc65c21fa8228dbc00a306a45471119b7d2d212c5cc5004efe2dccc1075b3 WatchSource:0}: Error finding container 475bc65c21fa8228dbc00a306a45471119b7d2d212c5cc5004efe2dccc1075b3: Status 404 returned error can't find the container with id 475bc65c21fa8228dbc00a306a45471119b7d2d212c5cc5004efe2dccc1075b3
Sep 11 17:39:43.974034 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.974013    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69bfb1df_6b20_422a_9264_bd89506ab8d4.slice/crio-98cbf9f1cb02a99657c5819cbfcefe517341e23efee9c905441798605a418e6f WatchSource:0}: Error finding container 98cbf9f1cb02a99657c5819cbfcefe517341e23efee9c905441798605a418e6f: Status 404 returned error can't find the container with id 98cbf9f1cb02a99657c5819cbfcefe517341e23efee9c905441798605a418e6f
Sep 11 17:39:43.990600 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:43.990577    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp"]
Sep 11 17:39:43.993118 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:43.993092    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod921d4454_3d1d_489b_a9d6_31f5e8af7dc5.slice/crio-933f437a5d02867711b62a2aa0e46eae02926d45bd8c0216a789fb75f39588d4 WatchSource:0}: Error finding container 933f437a5d02867711b62a2aa0e46eae02926d45bd8c0216a789fb75f39588d4: Status 404 returned error can't find the container with id 933f437a5d02867711b62a2aa0e46eae02926d45bd8c0216a789fb75f39588d4
Sep 11 17:39:44.042136 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.042112    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt"]
Sep 11 17:39:44.046340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.046217    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-545rn"]
Sep 11 17:39:44.048629 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:44.048593    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc070bb0_e27e_45c8_b7e5_987dabdbef97.slice/crio-585581bda181c13dc3bd9b4d0c38de357cb9e853b284b5a4aa18d565e357c8b4 WatchSource:0}: Error finding container 585581bda181c13dc3bd9b4d0c38de357cb9e853b284b5a4aa18d565e357c8b4: Status 404 returned error can't find the container with id 585581bda181c13dc3bd9b4d0c38de357cb9e853b284b5a4aa18d565e357c8b4
Sep 11 17:39:44.212083 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.212041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt" event={"ID":"79436815-d9f1-4115-a80d-c486e8f70f9c","Type":"ContainerStarted","Data":"a6c20a2ab4174c3802886c3544b7d2216d6ffa20c9fb9abe07a72d5736734ad0"}
Sep 11 17:39:44.213956 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.213912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" event={"ID":"d70ffa47-c8d2-491b-a8bc-550a503e9b3d","Type":"ContainerStarted","Data":"10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384"}
Sep 11 17:39:44.214087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.213961    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" event={"ID":"d70ffa47-c8d2-491b-a8bc-550a503e9b3d","Type":"ContainerStarted","Data":"90e2d7cbf758583e7c3dd979f74965a8a8b337de598ee53d8d929448b780cac3"}
Sep 11 17:39:44.214087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.214011    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:39:44.215391 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.215368    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7h22x" event={"ID":"197940f3-4a76-4c8f-bf3f-c4adcf446604","Type":"ContainerStarted","Data":"e2db1ad6e979b5f287a9697d1a62f2386de4352aface6a9148410c13eb91f625"}
Sep 11 17:39:44.216640 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.216621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cgzbn" event={"ID":"59f9e5b9-528f-4400-a787-3c339153c3b4","Type":"ContainerStarted","Data":"0aa7ae607cf31f4fdd03c437ca6f35c523b4f1bf89991ebb50a2b1bdb53e1d0f"}
Sep 11 17:39:44.217678 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.217657    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-545rn" event={"ID":"fc070bb0-e27e-45c8-b7e5-987dabdbef97","Type":"ContainerStarted","Data":"585581bda181c13dc3bd9b4d0c38de357cb9e853b284b5a4aa18d565e357c8b4"}
Sep 11 17:39:44.223950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.219723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rr858" event={"ID":"c9347efe-d030-4e8f-b60f-9929b060c87a","Type":"ContainerStarted","Data":"d36dd9fd727735e300c52ae739721aadde606d91daf52235fbb54a97c3cd60b0"}
Sep 11 17:39:44.223950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.219847    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rr858" event={"ID":"c9347efe-d030-4e8f-b60f-9929b060c87a","Type":"ContainerStarted","Data":"8b2cd93f246272dd2cc53420d8f9088f8d74ba736b78581247557d9a0e38ca3a"}
Sep 11 17:39:44.223950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.220454    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:39:44.226809 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.226774    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-89b8b6fbd-68pzd" event={"ID":"69bfb1df-6b20-422a-9264-bd89506ab8d4","Type":"ContainerStarted","Data":"6f19568cee941cf994849a2469edaf20664ce76660d817e6924bc0a938ea2c04"}
Sep 11 17:39:44.226899 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.226812    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-89b8b6fbd-68pzd" event={"ID":"69bfb1df-6b20-422a-9264-bd89506ab8d4","Type":"ContainerStarted","Data":"98cbf9f1cb02a99657c5819cbfcefe517341e23efee9c905441798605a418e6f"}
Sep 11 17:39:44.228160 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.228141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp" event={"ID":"921d4454-3d1d-489b-a9d6-31f5e8af7dc5","Type":"ContainerStarted","Data":"933f437a5d02867711b62a2aa0e46eae02926d45bd8c0216a789fb75f39588d4"}
Sep 11 17:39:44.229437 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.229417    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf" event={"ID":"041f6f00-15f9-48cf-82a2-fe253ff1f70a","Type":"ContainerStarted","Data":"475bc65c21fa8228dbc00a306a45471119b7d2d212c5cc5004efe2dccc1075b3"}
Sep 11 17:39:44.239531 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.239483    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" podStartSLOduration=66.239465609 podStartE2EDuration="1m6.239465609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:44.238409247 +0000 UTC m=+67.108452191" watchObservedRunningTime="2026-09-11 17:39:44.239465609 +0000 UTC m=+67.109508555"
Sep 11 17:39:44.255272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.255222    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rr858" podStartSLOduration=66.255206413 podStartE2EDuration="1m6.255206413s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:44.254165459 +0000 UTC m=+67.124208405" watchObservedRunningTime="2026-09-11 17:39:44.255206413 +0000 UTC m=+67.125249360"
Sep 11 17:39:44.267821 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.267790    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:44.276083 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.275737    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-89b8b6fbd-68pzd" podStartSLOduration=37.275719644 podStartE2EDuration="37.275719644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:44.275671593 +0000 UTC m=+67.145714538" watchObservedRunningTime="2026-09-11 17:39:44.275719644 +0000 UTC m=+67.145762581"
Sep 11 17:39:44.331721 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.331668    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:44.658392 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.658306    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:44.661379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.661337    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:44.873748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:44.873715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:45.237063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.237007    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:45.240025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.240002    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-89b8b6fbd-68pzd"
Sep 11 17:39:45.287903 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.287867    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"]
Sep 11 17:39:45.294915 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.294885    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"]
Sep 11 17:39:45.295744 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.295717    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.305272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.301670    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-7fqxw\""
Sep 11 17:39:45.305272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.301880    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 17:39:45.305272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.302919    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"]
Sep 11 17:39:45.307643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.305628    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 17:39:45.307643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.305868    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 17:39:45.307643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.306093    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 17:39:45.307643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.306948    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.307643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.307061    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"]
Sep 11 17:39:45.307643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.307174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.314555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.311733    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Sep 11 17:39:45.314555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.311893    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"]
Sep 11 17:39:45.314555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.311957    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 17:39:45.314555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.312052    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 17:39:45.314555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.312252    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 17:39:45.314555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.312334    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 17:39:45.327329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.325722    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"]
Sep 11 17:39:45.330749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.330707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:45.406908 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.405586    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-qmqz9"]
Sep 11 17:39:45.409826 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.409093    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416135    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416162    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d19688ad-8a00-4377-9c0f-932bd648996b-tmp\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416195    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d19688ad-8a00-4377-9c0f-932bd648996b-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416227    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-hub\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416252    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-ca\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416313    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz568\" (UniqueName: \"kubernetes.io/projected/f4c46156-32b6-4329-bb14-0dd100818ed7-kube-api-access-hz568\") pod \"managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6\" (UID: \"f4c46156-32b6-4329-bb14-0dd100818ed7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2zv6\" (UniqueName: \"kubernetes.io/projected/d19688ad-8a00-4377-9c0f-932bd648996b-kube-api-access-d2zv6\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48hf2\" (UniqueName: \"kubernetes.io/projected/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-kube-api-access-48hf2\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.416604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.416434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4c46156-32b6-4329-bb14-0dd100818ed7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6\" (UID: \"f4c46156-32b6-4329-bb14-0dd100818ed7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.418716 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.418697    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 11 17:39:45.420116 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.419953    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 11 17:39:45.425983 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.425961    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-cddvb\""
Sep 11 17:39:45.458979 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.458663    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-qmqz9"]
Sep 11 17:39:45.512374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.512265    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:45.517093 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-hub\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.517228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-ca\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.517228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517147    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.517228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517174    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz568\" (UniqueName: \"kubernetes.io/projected/f4c46156-32b6-4329-bb14-0dd100818ed7-kube-api-access-hz568\") pod \"managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6\" (UID: \"f4c46156-32b6-4329-bb14-0dd100818ed7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.517228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2zv6\" (UniqueName: \"kubernetes.io/projected/d19688ad-8a00-4377-9c0f-932bd648996b-kube-api-access-d2zv6\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.517452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517245    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48hf2\" (UniqueName: \"kubernetes.io/projected/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-kube-api-access-48hf2\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.517452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4c46156-32b6-4329-bb14-0dd100818ed7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6\" (UID: \"f4c46156-32b6-4329-bb14-0dd100818ed7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.517452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7g6p\" (UniqueName: \"kubernetes.io/projected/0579203d-ba0e-40e8-9756-6942bed7ec3a-kube-api-access-z7g6p\") pod \"downloads-76f99584f4-qmqz9\" (UID: \"0579203d-ba0e-40e8-9756-6942bed7ec3a\") " pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:39:45.517452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517391    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.517452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.517452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517441    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d19688ad-8a00-4377-9c0f-932bd648996b-tmp\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.517736 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.517477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d19688ad-8a00-4377-9c0f-932bd648996b-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.520579 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.519434    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.520579 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.520512    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d19688ad-8a00-4377-9c0f-932bd648996b-tmp\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.524020 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.523976    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.535851 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.535769    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz568\" (UniqueName: \"kubernetes.io/projected/f4c46156-32b6-4329-bb14-0dd100818ed7-kube-api-access-hz568\") pod \"managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6\" (UID: \"f4c46156-32b6-4329-bb14-0dd100818ed7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.537402 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.537333    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48hf2\" (UniqueName: \"kubernetes.io/projected/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-kube-api-access-48hf2\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.537803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.537756    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-ca\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.539244 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.538474    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-hub\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.539244 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.539033    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/d19688ad-8a00-4377-9c0f-932bd648996b-klusterlet-config\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.539671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.539629    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2zv6\" (UniqueName: \"kubernetes.io/projected/d19688ad-8a00-4377-9c0f-932bd648996b-kube-api-access-d2zv6\") pod \"klusterlet-addon-workmgr-6b9d68cd5c-nlwjm\" (UID: \"d19688ad-8a00-4377-9c0f-932bd648996b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.540205 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.540137    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4c46156-32b6-4329-bb14-0dd100818ed7-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6\" (UID: \"f4c46156-32b6-4329-bb14-0dd100818ed7\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.540781 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.540743    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/abdc2a00-0a5f-43f5-b98c-1d9afce5936e-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6d9f4968f7-kk5x9\" (UID: \"abdc2a00-0a5f-43f5-b98c-1d9afce5936e\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.604145 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.604091    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:45.618494 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.618168    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7g6p\" (UniqueName: \"kubernetes.io/projected/0579203d-ba0e-40e8-9756-6942bed7ec3a-kube-api-access-z7g6p\") pod \"downloads-76f99584f4-qmqz9\" (UID: \"0579203d-ba0e-40e8-9756-6942bed7ec3a\") " pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:39:45.630197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.630108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7g6p\" (UniqueName: \"kubernetes.io/projected/0579203d-ba0e-40e8-9756-6942bed7ec3a-kube-api-access-z7g6p\") pod \"downloads-76f99584f4-qmqz9\" (UID: \"0579203d-ba0e-40e8-9756-6942bed7ec3a\") " pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:39:45.634689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.634663    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"
Sep 11 17:39:45.645795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.645768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:45.653752 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.653727    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"
Sep 11 17:39:45.730398 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.730363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:39:45.802942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.802813    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:45.905193 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:45.905127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:46.003729 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:46.003578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:46.781208 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:46.781162    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:46.848694 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:46.848656    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:49.862447 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:49.862413    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9"]
Sep 11 17:39:49.871229 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:49.871174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6"]
Sep 11 17:39:49.880711 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:49.878320    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf4c46156_32b6_4329_bb14_0dd100818ed7.slice/crio-a4f3e3413fdf9a7ca7bf551b48e7408bef998b73bfbee8f2f250165edc987c6d WatchSource:0}: Error finding container a4f3e3413fdf9a7ca7bf551b48e7408bef998b73bfbee8f2f250165edc987c6d: Status 404 returned error can't find the container with id a4f3e3413fdf9a7ca7bf551b48e7408bef998b73bfbee8f2f250165edc987c6d
Sep 11 17:39:49.880711 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:49.880585    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"]
Sep 11 17:39:49.884952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:49.884810    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-qmqz9"]
Sep 11 17:39:49.892696 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:49.892669    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19688ad_8a00_4377_9c0f_932bd648996b.slice/crio-3515c0d000a15eeff51f1820a9360b230d7c342744e453124c09301f1502a2d1 WatchSource:0}: Error finding container 3515c0d000a15eeff51f1820a9360b230d7c342744e453124c09301f1502a2d1: Status 404 returned error can't find the container with id 3515c0d000a15eeff51f1820a9360b230d7c342744e453124c09301f1502a2d1
Sep 11 17:39:49.894976 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:49.894279    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0579203d_ba0e_40e8_9756_6942bed7ec3a.slice/crio-508407eb3e39f943286d684eaf5f1a2fcd57fc159f573aaaf7506f63996b8ccd WatchSource:0}: Error finding container 508407eb3e39f943286d684eaf5f1a2fcd57fc159f573aaaf7506f63996b8ccd: Status 404 returned error can't find the container with id 508407eb3e39f943286d684eaf5f1a2fcd57fc159f573aaaf7506f63996b8ccd
Sep 11 17:39:50.078414 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.078310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.173509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.173482    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.263888 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.263806    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7h22x" event={"ID":"197940f3-4a76-4c8f-bf3f-c4adcf446604","Type":"ContainerStarted","Data":"5e79d3ce83739ae3504727c00d6597df66ce36cd939c79a8122d65fafcb721dd"}
Sep 11 17:39:50.263888 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.263874    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-7h22x" event={"ID":"197940f3-4a76-4c8f-bf3f-c4adcf446604","Type":"ContainerStarted","Data":"bc1b4e6cd162ae057eacd7263693ae095b4f8d304ec7e0d35aa6124e8d036f95"}
Sep 11 17:39:50.265598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.265561    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-cgzbn" event={"ID":"59f9e5b9-528f-4400-a787-3c339153c3b4","Type":"ContainerStarted","Data":"9752b664ce8e8f6cb40e856c3eadfee97facdaeef22f5041025a4c0990bd1bd9"}
Sep 11 17:39:50.268656 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.268615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-545rn" event={"ID":"fc070bb0-e27e-45c8-b7e5-987dabdbef97","Type":"ContainerStarted","Data":"f5a88f63ecfac42995b469c7096c4bc4c36d5395b5133c144b1cb9ff2cb6883a"}
Sep 11 17:39:50.268789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.268691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-545rn" event={"ID":"fc070bb0-e27e-45c8-b7e5-987dabdbef97","Type":"ContainerStarted","Data":"2fcdfe626361ab2850e128acf9ab63e0749f67688c625061e083944139b64dfb"}
Sep 11 17:39:50.268789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.268766    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-545rn"
Sep 11 17:39:50.270598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.270577    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp" event={"ID":"921d4454-3d1d-489b-a9d6-31f5e8af7dc5","Type":"ContainerStarted","Data":"1032170b0b8967695aabf99169a13d734dd1108b1169bf7f0d5a5cf8575f0691"}
Sep 11 17:39:50.271801 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.271772    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-qmqz9" event={"ID":"0579203d-ba0e-40e8-9756-6942bed7ec3a","Type":"ContainerStarted","Data":"508407eb3e39f943286d684eaf5f1a2fcd57fc159f573aaaf7506f63996b8ccd"}
Sep 11 17:39:50.272839 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.272812    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6" event={"ID":"f4c46156-32b6-4329-bb14-0dd100818ed7","Type":"ContainerStarted","Data":"a4f3e3413fdf9a7ca7bf551b48e7408bef998b73bfbee8f2f250165edc987c6d"}
Sep 11 17:39:50.273901 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.273878    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9" event={"ID":"abdc2a00-0a5f-43f5-b98c-1d9afce5936e","Type":"ContainerStarted","Data":"ac96f7916d0b0c1c0108787ceafbc0865eda3da6a41b8645b7db641285d43a2d"}
Sep 11 17:39:50.275367 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.275326    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf" event={"ID":"041f6f00-15f9-48cf-82a2-fe253ff1f70a","Type":"ContainerStarted","Data":"9f69723ab128a85339dbb8d7a0d6b7fa347e414fa6986ed16a43ce667fff50c9"}
Sep 11 17:39:50.276420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.276397    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm" event={"ID":"d19688ad-8a00-4377-9c0f-932bd648996b","Type":"ContainerStarted","Data":"3515c0d000a15eeff51f1820a9360b230d7c342744e453124c09301f1502a2d1"}
Sep 11 17:39:50.278000 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.277982    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt" event={"ID":"79436815-d9f1-4115-a80d-c486e8f70f9c","Type":"ContainerStarted","Data":"19bf8800f103e370de31f887aa4063ad829c92a1c42038e4508f619611961423"}
Sep 11 17:39:50.278074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.278016    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt" event={"ID":"79436815-d9f1-4115-a80d-c486e8f70f9c","Type":"ContainerStarted","Data":"757f2fbdd053efdd1e76dcb0bbe4a5e38313ca04e72bba9979506d960c453e37"}
Sep 11 17:39:50.296532 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.296460    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-7h22x" podStartSLOduration=66.665513691 podStartE2EDuration="1m12.296443397s" totalImagesPullingTime="5.630929706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.295653478 +0000 UTC m=+73.165696425" watchObservedRunningTime="2026-09-11 17:39:50.296443397 +0000 UTC m=+73.166486343"
Sep 11 17:39:50.315111 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.315061    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.409718 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.409631    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-545rn" podStartSLOduration=33.874284717 podStartE2EDuration="39.409607794s" totalImagesPullingTime="5.535323077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.368006585 +0000 UTC m=+73.238049531" watchObservedRunningTime="2026-09-11 17:39:50.409607794 +0000 UTC m=+73.279650740"
Sep 11 17:39:50.410248 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.410219    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-v67mt" podStartSLOduration=37.917937042 podStartE2EDuration="43.410210167s" totalImagesPullingTime="5.492273125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.409066703 +0000 UTC m=+73.279109649" watchObservedRunningTime="2026-09-11 17:39:50.410210167 +0000 UTC m=+73.280253109"
Sep 11 17:39:50.432654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.432601    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cd954b897-p6tmp" podStartSLOduration=37.847008469 podStartE2EDuration="43.432586603s" totalImagesPullingTime="5.585578134s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.430716234 +0000 UTC m=+73.300759179" watchObservedRunningTime="2026-09-11 17:39:50.432586603 +0000 UTC m=+73.302629549"
Sep 11 17:39:50.440984 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.440960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.447318 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.447272    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-cgzbn" podStartSLOduration=66.697261957 podStartE2EDuration="1m12.447254339s" totalImagesPullingTime="5.749992382s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 17:38:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.446332047 +0000 UTC m=+73.316374993" watchObservedRunningTime="2026-09-11 17:39:50.447254339 +0000 UTC m=+73.317297285"
Sep 11 17:39:50.465906 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.465864    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-qwtvf" podStartSLOduration=37.854955483 podStartE2EDuration="43.465846802s" totalImagesPullingTime="5.610891319s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:50.465244127 +0000 UTC m=+73.335287072" watchObservedRunningTime="2026-09-11 17:39:50.465846802 +0000 UTC m=+73.335889747"
Sep 11 17:39:50.537200 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.537123    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.638387 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.638338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:50.739830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:50.739787    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:52.511181 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:52.511014    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:52.581472 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:52.581415    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.077459 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:53.077418    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.178909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:53.178848    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.426790 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:53.426628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.524848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:53.524799    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:53.982368 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:53.982165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:54.135962 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:54.135906    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:54.220925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:54.220855    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:55.105463 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:55.105407    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:55.164944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:55.164887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:55.740313 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:55.740256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:55.806884 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:55.806767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:56.425596 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:56.425501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:57.894637 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:57.894602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:57.996004 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:57.995965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:58.330104 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.330072    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm" event={"ID":"d19688ad-8a00-4377-9c0f-932bd648996b","Type":"ContainerStarted","Data":"cdf7710100773bbb45c5e92563e2883b29a6b98c929f29d95e32aeff9bc5f792"}
Sep 11 17:39:58.330228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.330155    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:58.331925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.331903    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm"
Sep 11 17:39:58.332409 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.332379    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6" event={"ID":"f4c46156-32b6-4329-bb14-0dd100818ed7","Type":"ContainerStarted","Data":"c09ee2dd7bcd4ce68207038eb4f823130160eb6cca1e3fd92c0f40572e0788b1"}
Sep 11 17:39:58.335021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.334993    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9" event={"ID":"abdc2a00-0a5f-43f5-b98c-1d9afce5936e","Type":"ContainerStarted","Data":"cf2bc13d485d4297739d481670e9556403f156560ab8b9d2e6fb156a00932e91"}
Sep 11 17:39:58.335118 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.335034    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9" event={"ID":"abdc2a00-0a5f-43f5-b98c-1d9afce5936e","Type":"ContainerStarted","Data":"30e1721b8655e6783b6258303335827ee5bc28126b86e0ec9158e0a8a4ff914e"}
Sep 11 17:39:58.335118 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.335052    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9" event={"ID":"abdc2a00-0a5f-43f5-b98c-1d9afce5936e","Type":"ContainerStarted","Data":"9c7d83aed3b28d35373f5bd17d7d932ff2b48b016e0dd08e889d367b93642a78"}
Sep 11 17:39:58.346503 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.346460    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-6b9d68cd5c-nlwjm" podStartSLOduration=5.159293165 podStartE2EDuration="13.346442873s" totalImagesPullingTime="8.187149708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:58.345434573 +0000 UTC m=+81.215477515" watchObservedRunningTime="2026-09-11 17:39:58.346442873 +0000 UTC m=+81.216485820"
Sep 11 17:39:58.362765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.362711    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84b8f9dc8-gfcr6" podStartSLOduration=5.183870124 podStartE2EDuration="13.362692839s" totalImagesPullingTime="8.178822715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:58.360912626 +0000 UTC m=+81.230955571" watchObservedRunningTime="2026-09-11 17:39:58.362692839 +0000 UTC m=+81.232735781"
Sep 11 17:39:58.401329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.401273    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6d9f4968f7-kk5x9" podStartSLOduration=5.214545116 podStartE2EDuration="13.401243701s" totalImagesPullingTime="8.186698585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:39:58.40106936 +0000 UTC m=+81.271112306" watchObservedRunningTime="2026-09-11 17:39:58.401243701 +0000 UTC m=+81.271286648"
Sep 11 17:39:58.531840 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.531804    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:58.762714 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.762584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:58.810606 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.810516    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9vxts"]
Sep 11 17:39:58.815776 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.815752    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.818082 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.817918    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 17:39:58.818082 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.817939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-m696x\""
Sep 11 17:39:58.819073 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.818729    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 17:39:58.819073 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.818933    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 17:39:58.819712 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.819455    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 17:39:58.875206 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.875153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:58.940317 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.940272    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941079 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941033    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-root\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941172 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941110    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thwwn\" (UniqueName: \"kubernetes.io/projected/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-kube-api-access-thwwn\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941172 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941157    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-sys\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941275 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941286    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-textfile\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941310    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-metrics-client-ca\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941456 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941366    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-tls\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.941456 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.941403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-wtmp\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:58.966953 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:58.966912    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.041845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.041806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-textfile\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.041856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-metrics-client-ca\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.041892    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-tls\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.041924    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-wtmp\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.041967    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.041993    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-root\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042025    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thwwn\" (UniqueName: \"kubernetes.io/projected/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-kube-api-access-thwwn\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042338 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042055    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-sys\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042338 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042097    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042338 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042311    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-wtmp\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042593 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042567    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-metrics-client-ca\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042670 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042640    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-root\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042725 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042687    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-sys\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.042908 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.042885    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-textfile\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.043069 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.043049    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-accelerators-collector-config\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.045249 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.045205    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.046000 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.045960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-node-exporter-tls\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.049960 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.049941    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thwwn\" (UniqueName: \"kubernetes.io/projected/bab029ac-f4d5-400f-9b28-bbbf0b17a5f4-kube-api-access-thwwn\") pod \"node-exporter-9vxts\" (UID: \"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4\") " pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.079240 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.079176    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.129571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.129530    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9vxts"
Sep 11 17:39:59.145248 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:39:59.145222    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbab029ac_f4d5_400f_9b28_bbbf0b17a5f4.slice/crio-30e16027f56ac5efe06097737d64b00b82699dcd1025726dd2198ce39bb4174e WatchSource:0}: Error finding container 30e16027f56ac5efe06097737d64b00b82699dcd1025726dd2198ce39bb4174e: Status 404 returned error can't find the container with id 30e16027f56ac5efe06097737d64b00b82699dcd1025726dd2198ce39bb4174e
Sep 11 17:39:59.259572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.259533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.344753 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.344639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9vxts" event={"ID":"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4","Type":"ContainerStarted","Data":"30e16027f56ac5efe06097737d64b00b82699dcd1025726dd2198ce39bb4174e"}
Sep 11 17:39:59.376149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.376113    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.506981 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.506898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.598334 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.598202    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.705889 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.705849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.803523 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.803469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:39:59.913541 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:39:59.913446    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:00.034745 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:00.034691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:00.109408 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:00.109375    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:00.189513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:00.189405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:00.255800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:00.255737    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:00.289621 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:00.289592    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-545rn"
Sep 11 17:40:00.327187 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:00.326812    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:01.359393 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:01.358547    2582 generic.go:356] "Generic (PLEG): container finished" podID="bab029ac-f4d5-400f-9b28-bbbf0b17a5f4" containerID="53c908a072fbf58b33d1780463f7af9268b0b0d9519d27756e941667c00cc0ef" exitCode=0
Sep 11 17:40:01.359393 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:01.358615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9vxts" event={"ID":"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4","Type":"ContainerDied","Data":"53c908a072fbf58b33d1780463f7af9268b0b0d9519d27756e941667c00cc0ef"}
Sep 11 17:40:01.785191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:01.785153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:01.906956 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:01.906911    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:02.196025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:02.195927    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:02.366732 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:02.366691    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9vxts" event={"ID":"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4","Type":"ContainerStarted","Data":"56bffb1a3d408958e6b83984ed4653b054d6dc60ecc1899f83bb6af8fc0d8ee2"}
Sep 11 17:40:02.367150 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:02.366755    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9vxts" event={"ID":"bab029ac-f4d5-400f-9b28-bbbf0b17a5f4","Type":"ContainerStarted","Data":"1cc544d2a2afa4e1f4b7c2630ae2f3bd580118752af5f3a8371725314f4784af"}
Sep 11 17:40:02.393412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:02.393326    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9vxts" podStartSLOduration=3.130133194 podStartE2EDuration="4.39330439s" totalImagesPullingTime="1.193608571s" totalInitContainerRuntime="69.562625ms" isStatefulPod=true podCreationTimestamp="2026-09-11 17:39:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:02.392785592 +0000 UTC m=+85.262828537" watchObservedRunningTime="2026-09-11 17:40:02.39330439 +0000 UTC m=+85.263347336"
Sep 11 17:40:02.395206 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:02.395178    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:02.487229 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:02.487183    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:03.573193 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:03.573158    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:03.604509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:03.604471    2582 patch_prober.go:28] interesting pod/image-registry-8bcc4bd74-qz5zs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 11 17:40:03.604681 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:03.604536    2582 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" podUID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:40:03.668887 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:03.668844    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:03.769789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:03.769754    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:04.115594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:04.115559    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:04.214368 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:04.214311    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:05.176063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:05.176026    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:05.241537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:05.241507    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:40:05.581151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:05.581090    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:05.648414 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:05.648288    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:08.053770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:08.053729    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bcc4bd74-qz5zs"]
Sep 11 17:40:08.256026 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:08.255986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:08.339976 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:08.339898    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:09.162756 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.162720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:09.265422 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.265392    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:09.401584 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.401541    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-qmqz9" event={"ID":"0579203d-ba0e-40e8-9756-6942bed7ec3a","Type":"ContainerStarted","Data":"ca0b0f0f040a8f2d6bae8b9a66cf5299559974d522aae8722689df4f7c639f04"}
Sep 11 17:40:09.401835 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.401812    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:40:09.403644 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.403616    2582 patch_prober.go:28] interesting pod/downloads-76f99584f4-qmqz9 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused" start-of-body=
Sep 11 17:40:09.403749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.403666    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-qmqz9" podUID="0579203d-ba0e-40e8-9756-6942bed7ec3a" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.25:8080/\": dial tcp 10.132.0.25:8080: connect: connection refused"
Sep 11 17:40:09.419080 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:09.419002    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-qmqz9" podStartSLOduration=5.106083378 podStartE2EDuration="24.418984046s" totalImagesPullingTime="19.312900668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:39:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:40:09.41679648 +0000 UTC m=+92.286839425" watchObservedRunningTime="2026-09-11 17:40:09.418984046 +0000 UTC m=+92.289026991"
Sep 11 17:40:10.425753 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:10.425715    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-qmqz9"
Sep 11 17:40:10.734733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:10.734692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:10.801555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:10.801517    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:13.241494 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:13.241456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:13.314464 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:13.314423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:14.823450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:14.823391    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:14.924610 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:14.924571    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:15.025396 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:15.025367    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:15.143799 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:15.143715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:15.244116 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:15.244045    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:16.251256 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:16.251224    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rr858"
Sep 11 17:40:17.509336 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:17.509298    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:17.614563 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:17.614522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:18.212620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:18.212578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:18.312758 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:18.312728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:19.918268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:19.918207    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:20.019233 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:20.019200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:22.320750 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:22.320706    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:22.421371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:22.421313    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:24.465572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:24.465543    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:24.564913 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:24.564870    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:24.666583 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:24.666534    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:25.746858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:25.746825    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:25.849117 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:25.849056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:32.499950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:32.499915    2582 generic.go:356] "Generic (PLEG): container finished" podID="e471abfe-aaf5-45bb-8f10-9ec40578bf95" containerID="6c5f1ebbd76638ccd574b891a9b5670d4afd2c411ea40a29e6237be6de4411c3" exitCode=0
Sep 11 17:40:32.500337 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:32.499986    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m" event={"ID":"e471abfe-aaf5-45bb-8f10-9ec40578bf95","Type":"ContainerDied","Data":"6c5f1ebbd76638ccd574b891a9b5670d4afd2c411ea40a29e6237be6de4411c3"}
Sep 11 17:40:32.500337 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:32.500318    2582 scope.go:117] "RemoveContainer" containerID="6c5f1ebbd76638ccd574b891a9b5670d4afd2c411ea40a29e6237be6de4411c3"
Sep 11 17:40:33.075099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.075047    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" podUID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" containerName="registry" containerID="cri-o://10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384" gracePeriod=30
Sep 11 17:40:33.344713 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.344651    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:40:33.436681 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436646    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-trusted-ca\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.436831 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436693    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fhrx\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-kube-api-access-9fhrx\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.436831 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436721    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-bound-sa-token\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.436831 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436746    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-ca-trust-extracted\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.436831 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436762    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-certificates\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.436831 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436796    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-installation-pull-secrets\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.437072 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436835    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-image-registry-private-configuration\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.437072 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.436903    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") pod \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\" (UID: \"d70ffa47-c8d2-491b-a8bc-550a503e9b3d\") "
Sep 11 17:40:33.437212 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.437183    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:40:33.437269 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.437247    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:40:33.439868 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.439835    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-kube-api-access-9fhrx" (OuterVolumeSpecName: "kube-api-access-9fhrx") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "kube-api-access-9fhrx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:40:33.440008 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.439876    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:33.440008 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.439905    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:40:33.440008 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.439989    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:40:33.440155 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.440059    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:40:33.445951 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.445927    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "d70ffa47-c8d2-491b-a8bc-550a503e9b3d" (UID: "d70ffa47-c8d2-491b-a8bc-550a503e9b3d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:40:33.505829 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.505800    2582 generic.go:356] "Generic (PLEG): container finished" podID="7a040f9a-b722-4aa6-9cf3-371f02ee172a" containerID="19868478c9547187fd03b892a7dad0eec8479c9f9ba6ea622a1263823f765615" exitCode=0
Sep 11 17:40:33.506167 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.505877    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ht8lt" event={"ID":"7a040f9a-b722-4aa6-9cf3-371f02ee172a","Type":"ContainerDied","Data":"19868478c9547187fd03b892a7dad0eec8479c9f9ba6ea622a1263823f765615"}
Sep 11 17:40:33.506272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.506254    2582 scope.go:117] "RemoveContainer" containerID="19868478c9547187fd03b892a7dad0eec8479c9f9ba6ea622a1263823f765615"
Sep 11 17:40:33.507305 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.507284    2582 generic.go:356] "Generic (PLEG): container finished" podID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" containerID="10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384" exitCode=0
Sep 11 17:40:33.507393 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.507314    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs"
Sep 11 17:40:33.507438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.507384    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" event={"ID":"d70ffa47-c8d2-491b-a8bc-550a503e9b3d","Type":"ContainerDied","Data":"10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384"}
Sep 11 17:40:33.507474 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.507443    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8bcc4bd74-qz5zs" event={"ID":"d70ffa47-c8d2-491b-a8bc-550a503e9b3d","Type":"ContainerDied","Data":"90e2d7cbf758583e7c3dd979f74965a8a8b337de598ee53d8d929448b780cac3"}
Sep 11 17:40:33.507513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.507472    2582 scope.go:117] "RemoveContainer" containerID="10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384"
Sep 11 17:40:33.509462 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.509441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6c8f84f897-xls2m" event={"ID":"e471abfe-aaf5-45bb-8f10-9ec40578bf95","Type":"ContainerStarted","Data":"1c6d83b4c5dbadef8ca99f4952acf598c3909acfd0758b021fbb8367f7522748"}
Sep 11 17:40:33.519053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.519035    2582 scope.go:117] "RemoveContainer" containerID="10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384"
Sep 11 17:40:33.519303 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:40:33.519283    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384\": container with ID starting with 10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384 not found: ID does not exist" containerID="10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384"
Sep 11 17:40:33.519388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.519309    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384"} err="failed to get container status \"10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384\": rpc error: code = NotFound desc = could not find container \"10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384\": container with ID starting with 10c7dd41e0b4317e7cbd9b9c253c6a04518bc30770c7158df3863279fc4a8384 not found: ID does not exist"
Sep 11 17:40:33.538286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538262    2582 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-tls\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538292    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-trusted-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538308    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fhrx\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-kube-api-access-9fhrx\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538325    2582 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-bound-sa-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538340    2582 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-ca-trust-extracted\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538370    2582 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-registry-certificates\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538381    2582 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-installation-pull-secrets\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.538410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.538396    2582 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d70ffa47-c8d2-491b-a8bc-550a503e9b3d-image-registry-private-configuration\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:40:33.547460 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.547437    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-8bcc4bd74-qz5zs"]
Sep 11 17:40:33.554153 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.554131    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-8bcc4bd74-qz5zs"]
Sep 11 17:40:33.745125 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:33.745095    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" path="/var/lib/kubelet/pods/d70ffa47-c8d2-491b-a8bc-550a503e9b3d/volumes"
Sep 11 17:40:34.188435 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:34.188398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:34.287892 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:34.287847    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:34.515711 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:34.515669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-ht8lt" event={"ID":"7a040f9a-b722-4aa6-9cf3-371f02ee172a","Type":"ContainerStarted","Data":"b8c77242c4d55a226e48b48d2268789bf601a19b6e104d22a9c03265c8d995c8"}
Sep 11 17:40:37.536906 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:37.536875    2582 generic.go:356] "Generic (PLEG): container finished" podID="73c2ef90-9bd5-4110-892d-296f12551de4" containerID="7a7e501a024976fe0620628adeb56064b68c9fe595275c49911538b4ad090e65" exitCode=0
Sep 11 17:40:37.537374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:37.536939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k" event={"ID":"73c2ef90-9bd5-4110-892d-296f12551de4","Type":"ContainerDied","Data":"7a7e501a024976fe0620628adeb56064b68c9fe595275c49911538b4ad090e65"}
Sep 11 17:40:37.537374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:37.537305    2582 scope.go:117] "RemoveContainer" containerID="7a7e501a024976fe0620628adeb56064b68c9fe595275c49911538b4ad090e65"
Sep 11 17:40:38.543251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:38.543212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-fqn8k" event={"ID":"73c2ef90-9bd5-4110-892d-296f12551de4","Type":"ContainerStarted","Data":"59ea9db062d37ba600cd29b1767bc11819b45d9c142e43eaf3d6f2cea4d24ae5"}
Sep 11 17:40:42.697168 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:42.697131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:50.847526 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:50.847487    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:52.948814 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:52.948778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:40:53.014525 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:40:53.014488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:14.923554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:14.923511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:15.022412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:15.022365    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:26.904018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:26.903963    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:26.980587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:26.980557    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:27.715196 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:27.715151    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:40.736328 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:40.736283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:40.837331 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:40.837299    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:40.901270 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:40.901225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:43.728076 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:43.728035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:43.797127 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:43.797092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:54.806825 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:54.806788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:54.907573 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:54.907539    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:41:54.996737 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:41:54.996699    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:28.595709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:42:28.595650    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:28.698706 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:42:28.698673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:55.504287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:42:55.504247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:56.343951 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:42:56.343915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:59.138765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:42:59.138724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:42:59.238313 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:42:59.238277    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:00.764939 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:00.764867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:00.868921 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:00.868885    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:01.310626 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:01.310588    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:01.428462 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:01.428432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:02.981898 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:02.981864    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:03.162423 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:03.162389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:06.093748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:06.093715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:06.190537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:06.190495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:08.700748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:08.700709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:08.801701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:08.801672    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:11.377419 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:11.377378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:17.692789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:17.692757    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:17.792519 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:17.792483    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:23.486570 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:23.486535    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:23.586794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:23.586754    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:25.634642 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:25.634608    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:37.544506 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:37.544472    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:43:37.546665 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:37.546641    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:43:37.556331 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:37.556301    2582 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 17:43:39.656064 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:39.656028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:40.677643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.677608    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"]
Sep 11 17:43:40.680457 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.678534    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" containerName="registry"
Sep 11 17:43:40.680457 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.678566    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" containerName="registry"
Sep 11 17:43:40.680457 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.678750    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d70ffa47-c8d2-491b-a8bc-550a503e9b3d" containerName="registry"
Sep 11 17:43:40.682761 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.682695    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.689828 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.689769    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 17:43:40.689898 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.689840    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 17:43:40.689943 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.689769    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 17:43:40.690056 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.690038    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-jvzqs\""
Sep 11 17:43:40.690111 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.690096    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 17:43:40.719435 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.719406    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"]
Sep 11 17:43:40.730696 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.730674    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75971c72-da6f-40e6-ac3b-ea87b6151779-webhook-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.730805 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.730703    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75971c72-da6f-40e6-ac3b-ea87b6151779-manifests\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.730805 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.730735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75971c72-da6f-40e6-ac3b-ea87b6151779-tmp\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.730805 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.730763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fps69\" (UniqueName: \"kubernetes.io/projected/75971c72-da6f-40e6-ac3b-ea87b6151779-kube-api-access-fps69\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.730805 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.730800    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75971c72-da6f-40e6-ac3b-ea87b6151779-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.831937 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.831911    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75971c72-da6f-40e6-ac3b-ea87b6151779-manifests\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.832103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.831953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75971c72-da6f-40e6-ac3b-ea87b6151779-tmp\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.832103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.831976    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fps69\" (UniqueName: \"kubernetes.io/projected/75971c72-da6f-40e6-ac3b-ea87b6151779-kube-api-access-fps69\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.832103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.832020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75971c72-da6f-40e6-ac3b-ea87b6151779-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.832103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.832050    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75971c72-da6f-40e6-ac3b-ea87b6151779-webhook-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.832443 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.832420    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/75971c72-da6f-40e6-ac3b-ea87b6151779-manifests\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.834126 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.834104    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/75971c72-da6f-40e6-ac3b-ea87b6151779-tmp\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.834369 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.834337    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/75971c72-da6f-40e6-ac3b-ea87b6151779-webhook-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.834515 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.834493    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/75971c72-da6f-40e6-ac3b-ea87b6151779-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.839614 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.839595    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fps69\" (UniqueName: \"kubernetes.io/projected/75971c72-da6f-40e6-ac3b-ea87b6151779-kube-api-access-fps69\") pod \"opendatahub-operator-controller-manager-86997d8dbf-m5wb2\" (UID: \"75971c72-da6f-40e6-ac3b-ea87b6151779\") " pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:40.993205 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:40.993181    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:41.006554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.006521    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:41.106601 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.106568    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:41.208090 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.208058    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:41.267041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.267016    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"]
Sep 11 17:43:41.268867 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:43:41.268841    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75971c72_da6f_40e6_ac3b_ea87b6151779.slice/crio-102e1b751d41003b3963b0586e3cf729454a655c1ba6feb36a8f97a43cc53e03 WatchSource:0}: Error finding container 102e1b751d41003b3963b0586e3cf729454a655c1ba6feb36a8f97a43cc53e03: Status 404 returned error can't find the container with id 102e1b751d41003b3963b0586e3cf729454a655c1ba6feb36a8f97a43cc53e03
Sep 11 17:43:41.270680 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.270662    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:43:41.312781 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.312756    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:41.409547 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:41.409509    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:42.239643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:42.239604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2" event={"ID":"75971c72-da6f-40e6-ac3b-ea87b6151779","Type":"ContainerStarted","Data":"102e1b751d41003b3963b0586e3cf729454a655c1ba6feb36a8f97a43cc53e03"}
Sep 11 17:43:45.254218 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:45.254188    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2" event={"ID":"75971c72-da6f-40e6-ac3b-ea87b6151779","Type":"ContainerStarted","Data":"e265a8dac582475df7d209bc617bd7dce9d36a800682dbcefb52f16e584cd133"}
Sep 11 17:43:46.264626 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:46.264587    2582 generic.go:356] "Generic (PLEG): container finished" podID="75971c72-da6f-40e6-ac3b-ea87b6151779" containerID="e265a8dac582475df7d209bc617bd7dce9d36a800682dbcefb52f16e584cd133" exitCode=0
Sep 11 17:43:46.265008 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:46.264635    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2" event={"ID":"75971c72-da6f-40e6-ac3b-ea87b6151779","Type":"ContainerDied","Data":"e265a8dac582475df7d209bc617bd7dce9d36a800682dbcefb52f16e584cd133"}
Sep 11 17:43:47.270987 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:47.270950    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2" event={"ID":"75971c72-da6f-40e6-ac3b-ea87b6151779","Type":"ContainerStarted","Data":"465851121aa09a311cb5b621200df95e03f8626da3882189f1f0ac9743318eb8"}
Sep 11 17:43:47.271370 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:47.271059    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:47.293897 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:47.293851    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2" podStartSLOduration=2.9176052649999997 podStartE2EDuration="7.293836164s" totalImagesPullingTime="3.294491146s" totalInitContainerRuntime="1.081739753s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:43:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:43:47.293040011 +0000 UTC m=+310.163082955" watchObservedRunningTime="2026-09-11 17:43:47.293836164 +0000 UTC m=+310.163879108"
Sep 11 17:43:58.278063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:58.278029    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86997d8dbf-m5wb2"
Sep 11 17:43:59.463206 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:59.463166    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:43:59.613702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:43:59.613673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:00.026990 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:00.026956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:00.222158 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:00.222108    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:05.700538 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.700498    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:05.715044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.715010    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"]
Sep 11 17:44:05.718220 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.718200    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.720980 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.720951    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 17:44:05.720980 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.720957    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 17:44:05.721116 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.721022    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-2dqgf\""
Sep 11 17:44:05.727527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.727503    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"]
Sep 11 17:44:05.822449 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.822421    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prbg6\" (UniqueName: \"kubernetes.io/projected/a9af83e1-74b6-4d75-a854-1d49d829d2da-kube-api-access-prbg6\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.822587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.822456    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9af83e1-74b6-4d75-a854-1d49d829d2da-tls-certs\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.822587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.822473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9af83e1-74b6-4d75-a854-1d49d829d2da-tmp\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.906747 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.906696    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:05.923649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.923626    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prbg6\" (UniqueName: \"kubernetes.io/projected/a9af83e1-74b6-4d75-a854-1d49d829d2da-kube-api-access-prbg6\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.923792 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.923654    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9af83e1-74b6-4d75-a854-1d49d829d2da-tls-certs\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.923792 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.923672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9af83e1-74b6-4d75-a854-1d49d829d2da-tmp\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.925895 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.925866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a9af83e1-74b6-4d75-a854-1d49d829d2da-tmp\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.926121 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.926097    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9af83e1-74b6-4d75-a854-1d49d829d2da-tls-certs\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:05.932191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:05.932157    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prbg6\" (UniqueName: \"kubernetes.io/projected/a9af83e1-74b6-4d75-a854-1d49d829d2da-kube-api-access-prbg6\") pod \"kube-auth-proxy-64d8557cb6-fwh9z\" (UID: \"a9af83e1-74b6-4d75-a854-1d49d829d2da\") " pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:06.028514 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:06.028487    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"
Sep 11 17:44:06.300538 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:06.300463    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z"]
Sep 11 17:44:06.303625 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:44:06.303599    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9af83e1_74b6_4d75_a854_1d49d829d2da.slice/crio-b9edd7920abbec0cb1f20695f26721057765746d7868e324f4ea5e67776164ab WatchSource:0}: Error finding container b9edd7920abbec0cb1f20695f26721057765746d7868e324f4ea5e67776164ab: Status 404 returned error can't find the container with id b9edd7920abbec0cb1f20695f26721057765746d7868e324f4ea5e67776164ab
Sep 11 17:44:06.346594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:06.346563    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z" event={"ID":"a9af83e1-74b6-4d75-a854-1d49d829d2da","Type":"ContainerStarted","Data":"b9edd7920abbec0cb1f20695f26721057765746d7868e324f4ea5e67776164ab"}
Sep 11 17:44:07.241174 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.241138    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"]
Sep 11 17:44:07.244649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.244623    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.247087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.246872    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rmfvs\""
Sep 11 17:44:07.247087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.246889    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 17:44:07.247087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.247054    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 17:44:07.247276 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.247181    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 17:44:07.247732 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.247697    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 17:44:07.253510 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.253165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:07.260274 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.260254    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"]
Sep 11 17:44:07.330844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.330811    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:07.335722 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335692    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.335864 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335774    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.335864 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b81fc57e-b328-439d-803d-ab1a65fe9cb1-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.335988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgk8s\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-kube-api-access-qgk8s\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.335988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.335988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335964    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.335988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.335983    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436750 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436716    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436750 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436753    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b81fc57e-b328-439d-803d-ab1a65fe9cb1-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436774    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgk8s\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-kube-api-access-qgk8s\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436801    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436836    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.436947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.436906    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.437594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.437560    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.439051 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.439023    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b81fc57e-b328-439d-803d-ab1a65fe9cb1-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.439230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.439209    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.439287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.439235    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.439287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.439261    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.445395 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.445367    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.445550 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.445534    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgk8s\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-kube-api-access-qgk8s\") pod \"istiod-openshift-gateway-7b7995d5cd-r2wv5\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.556701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.556608    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:07.847464 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:07.847387    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"]
Sep 11 17:44:07.851483 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:44:07.851455    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb81fc57e_b328_439d_803d_ab1a65fe9cb1.slice/crio-8db8c737c7734350ac6e9deb0ccff926cebaced6805aa5e5bb95af27e589875b WatchSource:0}: Error finding container 8db8c737c7734350ac6e9deb0ccff926cebaced6805aa5e5bb95af27e589875b: Status 404 returned error can't find the container with id 8db8c737c7734350ac6e9deb0ccff926cebaced6805aa5e5bb95af27e589875b
Sep 11 17:44:08.359668 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:08.359634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" event={"ID":"b81fc57e-b328-439d-803d-ab1a65fe9cb1","Type":"ContainerStarted","Data":"8db8c737c7734350ac6e9deb0ccff926cebaced6805aa5e5bb95af27e589875b"}
Sep 11 17:44:08.387844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:08.387799    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:10.371514 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:10.371466    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z" event={"ID":"a9af83e1-74b6-4d75-a854-1d49d829d2da","Type":"ContainerStarted","Data":"7a303e8fc41c63fccc7011a0057bb99efefb50d0239fccbb196dfdb701e88c19"}
Sep 11 17:44:10.388781 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:10.388725    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-64d8557cb6-fwh9z" podStartSLOduration=2.375542764 podStartE2EDuration="5.388708386s" totalImagesPullingTime="3.013165622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:44:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:44:10.387756187 +0000 UTC m=+333.257799134" watchObservedRunningTime="2026-09-11 17:44:10.388708386 +0000 UTC m=+333.258751334"
Sep 11 17:44:11.133055 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.133020    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:44:11.133165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.133087    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:44:11.207135 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.207102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.310207 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.310154    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.378395 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.378342    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" event={"ID":"b81fc57e-b328-439d-803d-ab1a65fe9cb1","Type":"ContainerStarted","Data":"1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74"}
Sep 11 17:44:11.411819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.411730    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" podStartSLOduration=1.132785633 podStartE2EDuration="4.41171428s" totalImagesPullingTime="3.278928647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:44:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:44:11.409646575 +0000 UTC m=+334.279689520" watchObservedRunningTime="2026-09-11 17:44:11.41171428 +0000 UTC m=+334.281757225"
Sep 11 17:44:11.420305 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.420274    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.563848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.563797    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.874266 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.874234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:11.997826 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:11.997792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:12.227335 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:12.227248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:12.384066 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:12.384037    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:12.385771 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:12.385738    2582 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-r2wv5 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:44:12.385889 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:12.385800    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" podUID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:44:13.055775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:13.055731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:13.389991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:13.389920    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:13.394270 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:13.394247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:13.494322 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:13.494293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.583838 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.583802    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.684900 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.684867    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"]
Sep 11 17:44:21.688984 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.688958    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.694772 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.694752    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 11 17:44:21.694905 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.694779    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-lqtc8\""
Sep 11 17:44:21.699061 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.699039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.708401 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.708379    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"]
Sep 11 17:44:21.754808 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.754780    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqctk\" (UniqueName: \"kubernetes.io/projected/f483e181-6ffb-4ba2-b50a-c622583c846b-kube-api-access-pqctk\") pod \"kserve-controller-manager-d6f4b65b8-pw7kj\" (UID: \"f483e181-6ffb-4ba2-b50a-c622583c846b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.754947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.754874    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f483e181-6ffb-4ba2-b50a-c622583c846b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pw7kj\" (UID: \"f483e181-6ffb-4ba2-b50a-c622583c846b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.802229 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.802184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.859444 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.856152    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f483e181-6ffb-4ba2-b50a-c622583c846b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pw7kj\" (UID: \"f483e181-6ffb-4ba2-b50a-c622583c846b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.859444 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.856263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqctk\" (UniqueName: \"kubernetes.io/projected/f483e181-6ffb-4ba2-b50a-c622583c846b-kube-api-access-pqctk\") pod \"kserve-controller-manager-d6f4b65b8-pw7kj\" (UID: \"f483e181-6ffb-4ba2-b50a-c622583c846b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.863012 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.862985    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f483e181-6ffb-4ba2-b50a-c622583c846b-cert\") pod \"kserve-controller-manager-d6f4b65b8-pw7kj\" (UID: \"f483e181-6ffb-4ba2-b50a-c622583c846b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.865932 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.865905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqctk\" (UniqueName: \"kubernetes.io/projected/f483e181-6ffb-4ba2-b50a-c622583c846b-kube-api-access-pqctk\") pod \"kserve-controller-manager-d6f4b65b8-pw7kj\" (UID: \"f483e181-6ffb-4ba2-b50a-c622583c846b\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:21.898915 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.898886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:21.998991 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:21.998952    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:22.031790 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.031755    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:22.159946 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.159835    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:22.276920 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.276876    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"]
Sep 11 17:44:22.279931 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:44:22.279907    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf483e181_6ffb_4ba2_b50a_c622583c846b.slice/crio-e1893bf000e2fd093b5e5e77e1bb758758bccc367e81a7d0c3a1ca54dc404651 WatchSource:0}: Error finding container e1893bf000e2fd093b5e5e77e1bb758758bccc367e81a7d0c3a1ca54dc404651: Status 404 returned error can't find the container with id e1893bf000e2fd093b5e5e77e1bb758758bccc367e81a7d0c3a1ca54dc404651
Sep 11 17:44:22.431365 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.431277    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj" event={"ID":"f483e181-6ffb-4ba2-b50a-c622583c846b","Type":"ContainerStarted","Data":"e1893bf000e2fd093b5e5e77e1bb758758bccc367e81a7d0c3a1ca54dc404651"}
Sep 11 17:44:22.473002 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.472970    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"]
Sep 11 17:44:22.473165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.473145    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" podUID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" containerName="discovery" containerID="cri-o://1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74" gracePeriod=30
Sep 11 17:44:22.698264 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.698237    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:22.763602 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763573    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-cacerts\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.763602 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763607    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-ca-configmap\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.763803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763635    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-dns-cert\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.763803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763657    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b81fc57e-b328-439d-803d-ab1a65fe9cb1-local-certs\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.763803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763688    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-token\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.763803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763704    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-kubeconfig\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.763803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.763725    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgk8s\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-kube-api-access-qgk8s\") pod \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\" (UID: \"b81fc57e-b328-439d-803d-ab1a65fe9cb1\") "
Sep 11 17:44:22.764073 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.764000    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:44:22.765929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.765898    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:22.766045 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.765956    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:22.766188 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.766162    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:22.766252 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.766230    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b81fc57e-b328-439d-803d-ab1a65fe9cb1-local-certs" (OuterVolumeSpecName: "local-certs") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:44:22.766289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.766260    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-cacerts" (OuterVolumeSpecName: "cacerts") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:44:22.766323 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.766301    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-kube-api-access-qgk8s" (OuterVolumeSpecName: "kube-api-access-qgk8s") pod "b81fc57e-b328-439d-803d-ab1a65fe9cb1" (UID: "b81fc57e-b328-439d-803d-ab1a65fe9cb1"). InnerVolumeSpecName "kube-api-access-qgk8s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864503    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-ca-configmap\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864522    2582 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-csr-dns-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864531    2582 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/b81fc57e-b328-439d-803d-ab1a65fe9cb1-local-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864539    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864547    2582 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-istio-kubeconfig\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864555    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgk8s\" (UniqueName: \"kubernetes.io/projected/b81fc57e-b328-439d-803d-ab1a65fe9cb1-kube-api-access-qgk8s\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:22.864594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:22.864564    2582 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/b81fc57e-b328-439d-803d-ab1a65fe9cb1-cacerts\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:44:23.438470 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.438437    2582 generic.go:356] "Generic (PLEG): container finished" podID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" containerID="1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74" exitCode=0
Sep 11 17:44:23.438654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.438495    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"
Sep 11 17:44:23.438654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.438517    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" event={"ID":"b81fc57e-b328-439d-803d-ab1a65fe9cb1","Type":"ContainerDied","Data":"1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74"}
Sep 11 17:44:23.438654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.438566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5" event={"ID":"b81fc57e-b328-439d-803d-ab1a65fe9cb1","Type":"ContainerDied","Data":"8db8c737c7734350ac6e9deb0ccff926cebaced6805aa5e5bb95af27e589875b"}
Sep 11 17:44:23.438654 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.438592    2582 scope.go:117] "RemoveContainer" containerID="1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74"
Sep 11 17:44:23.449191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.449170    2582 scope.go:117] "RemoveContainer" containerID="1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74"
Sep 11 17:44:23.449510 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:44:23.449482    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74\": container with ID starting with 1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74 not found: ID does not exist" containerID="1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74"
Sep 11 17:44:23.449617 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.449517    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74"} err="failed to get container status \"1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74\": rpc error: code = NotFound desc = could not find container \"1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74\": container with ID starting with 1dbd353f736f5d588aa2563e59273ae4f7c4658630d7fa42166e5a3c8156ec74 not found: ID does not exist"
Sep 11 17:44:23.465715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.465693    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"]
Sep 11 17:44:23.478250 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.478233    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r2wv5"]
Sep 11 17:44:23.746266 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:23.746237    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" path="/var/lib/kubelet/pods/b81fc57e-b328-439d-803d-ab1a65fe9cb1/volumes"
Sep 11 17:44:25.449945 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:25.449906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj" event={"ID":"f483e181-6ffb-4ba2-b50a-c622583c846b","Type":"ContainerStarted","Data":"02d2838fc9480630ff907ca36387429ee77b2e2e541cc445322184c7ec53240d"}
Sep 11 17:44:25.450395 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:25.449986    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:25.471307 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:25.471262    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj" podStartSLOduration=1.8837409489999999 podStartE2EDuration="4.471247741s" totalImagesPullingTime="2.587506792s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:44:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:44:25.469798872 +0000 UTC m=+348.339841816" watchObservedRunningTime="2026-09-11 17:44:25.471247741 +0000 UTC m=+348.341290687"
Sep 11 17:44:45.058172 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:45.058133    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:56.460063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:56.460033    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-pw7kj"
Sep 11 17:44:58.969415 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:58.969377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:44:59.070737 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:44:59.070695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:01.743805 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:01.743757    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:01.807629 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:01.807595    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:12.300733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:12.300695    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:12.411204 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:12.411159    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:12.512710 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:12.512673    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:12.610533 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:12.610444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:19.686014 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:19.685978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:19.741970 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:19.741930    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:29.056550 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:29.056515    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:31.223509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.223479    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"]
Sep 11 17:45:31.223857 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.223803    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" containerName="discovery"
Sep 11 17:45:31.223857 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.223815    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" containerName="discovery"
Sep 11 17:45:31.223957 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.223886    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b81fc57e-b328-439d-803d-ab1a65fe9cb1" containerName="discovery"
Sep 11 17:45:31.226846 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.226830    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:31.229289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.229260    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 17:45:31.229448 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.229342    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-djp7d\""
Sep 11 17:45:31.229621 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.229607    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 17:45:31.229927 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.229910    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 17:45:31.240814 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.240791    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"]
Sep 11 17:45:31.383128 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.383094    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfrl2\" (UniqueName: \"kubernetes.io/projected/26d8b7e2-8501-47f8-8e5c-9056b155a849-kube-api-access-vfrl2\") pod \"dns-operator-controller-manager-7b84cd7c9c-wzvc8\" (UID: \"26d8b7e2-8501-47f8-8e5c-9056b155a849\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:31.484037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.484007    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfrl2\" (UniqueName: \"kubernetes.io/projected/26d8b7e2-8501-47f8-8e5c-9056b155a849-kube-api-access-vfrl2\") pod \"dns-operator-controller-manager-7b84cd7c9c-wzvc8\" (UID: \"26d8b7e2-8501-47f8-8e5c-9056b155a849\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:31.491819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.491793    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfrl2\" (UniqueName: \"kubernetes.io/projected/26d8b7e2-8501-47f8-8e5c-9056b155a849-kube-api-access-vfrl2\") pod \"dns-operator-controller-manager-7b84cd7c9c-wzvc8\" (UID: \"26d8b7e2-8501-47f8-8e5c-9056b155a849\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:31.536836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.536811    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:31.814294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.814158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"]
Sep 11 17:45:31.818368 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:31.818309    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d8b7e2_8501_47f8_8e5c_9056b155a849.slice/crio-e0d233ca8d642b866223ec2c49f301886f9b3cae4aa227e91bbf9c56048523cd WatchSource:0}: Error finding container e0d233ca8d642b866223ec2c49f301886f9b3cae4aa227e91bbf9c56048523cd: Status 404 returned error can't find the container with id e0d233ca8d642b866223ec2c49f301886f9b3cae4aa227e91bbf9c56048523cd
Sep 11 17:45:31.854606 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:31.854572    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:32.734065 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:32.734030    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8" event={"ID":"26d8b7e2-8501-47f8-8e5c-9056b155a849","Type":"ContainerStarted","Data":"e0d233ca8d642b866223ec2c49f301886f9b3cae4aa227e91bbf9c56048523cd"}
Sep 11 17:45:33.240002 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.239973    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"]
Sep 11 17:45:33.244421 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.244402    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:33.247050 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.247028    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-b5q2g\""
Sep 11 17:45:33.263830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.263810    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"]
Sep 11 17:45:33.398263 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.398229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8jzx\" (UniqueName: \"kubernetes.io/projected/d1d5f3d1-1641-44ba-a8c6-470a28043efa-kube-api-access-x8jzx\") pod \"limitador-operator-controller-manager-74495c69f8-2vddc\" (UID: \"d1d5f3d1-1641-44ba-a8c6-470a28043efa\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:33.499728 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.499637    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8jzx\" (UniqueName: \"kubernetes.io/projected/d1d5f3d1-1641-44ba-a8c6-470a28043efa-kube-api-access-x8jzx\") pod \"limitador-operator-controller-manager-74495c69f8-2vddc\" (UID: \"d1d5f3d1-1641-44ba-a8c6-470a28043efa\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:33.509342 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.509314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8jzx\" (UniqueName: \"kubernetes.io/projected/d1d5f3d1-1641-44ba-a8c6-470a28043efa-kube-api-access-x8jzx\") pod \"limitador-operator-controller-manager-74495c69f8-2vddc\" (UID: \"d1d5f3d1-1641-44ba-a8c6-470a28043efa\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:33.554836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.554801    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:33.846007 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.845971    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"]
Sep 11 17:45:33.848488 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:33.848455    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d5f3d1_1641_44ba_a8c6_470a28043efa.slice/crio-5009159c0ba088093a372ae1eb389b7cd47f5365ef1ceffc8e5182e9f2dfa2bd WatchSource:0}: Error finding container 5009159c0ba088093a372ae1eb389b7cd47f5365ef1ceffc8e5182e9f2dfa2bd: Status 404 returned error can't find the container with id 5009159c0ba088093a372ae1eb389b7cd47f5365ef1ceffc8e5182e9f2dfa2bd
Sep 11 17:45:33.868916 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.868886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:33.966010 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:33.965968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:34.110544 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:34.110454    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:34.745431 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:34.745400    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc" event={"ID":"d1d5f3d1-1641-44ba-a8c6-470a28043efa","Type":"ContainerStarted","Data":"5009159c0ba088093a372ae1eb389b7cd47f5365ef1ceffc8e5182e9f2dfa2bd"}
Sep 11 17:45:35.505914 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.505883    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:35.751506 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.751476    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8" event={"ID":"26d8b7e2-8501-47f8-8e5c-9056b155a849","Type":"ContainerStarted","Data":"136989fdb0edce9982fa4fdde69e6711636dcc8da5a293957efd5741d7de8c16"}
Sep 11 17:45:35.751668 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.751606    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:35.775280 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.775194    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8" podStartSLOduration=1.412176404 podStartE2EDuration="4.775178848s" totalImagesPullingTime="3.363002444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:35.773826143 +0000 UTC m=+418.643869090" watchObservedRunningTime="2026-09-11 17:45:35.775178848 +0000 UTC m=+418.645221792"
Sep 11 17:45:35.860713 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.860682    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-gbxbq"]
Sep 11 17:45:35.863858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.863841    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:35.866714 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.866696    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-nk4sl\""
Sep 11 17:45:35.879578 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:35.879552    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-gbxbq"]
Sep 11 17:45:36.021628 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.021598    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl96f\" (UniqueName: \"kubernetes.io/projected/0ce262a3-df86-49d9-ab72-6eab8d280c04-kube-api-access-jl96f\") pod \"authorino-operator-76b478bd78-gbxbq\" (UID: \"0ce262a3-df86-49d9-ab72-6eab8d280c04\") " pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:36.123045 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.122971    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl96f\" (UniqueName: \"kubernetes.io/projected/0ce262a3-df86-49d9-ab72-6eab8d280c04-kube-api-access-jl96f\") pod \"authorino-operator-76b478bd78-gbxbq\" (UID: \"0ce262a3-df86-49d9-ab72-6eab8d280c04\") " pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:36.132337 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.132309    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl96f\" (UniqueName: \"kubernetes.io/projected/0ce262a3-df86-49d9-ab72-6eab8d280c04-kube-api-access-jl96f\") pod \"authorino-operator-76b478bd78-gbxbq\" (UID: \"0ce262a3-df86-49d9-ab72-6eab8d280c04\") " pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:36.173676 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.173649    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:36.448464 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.448432    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-gbxbq"]
Sep 11 17:45:36.451096 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:36.451067    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ce262a3_df86_49d9_ab72_6eab8d280c04.slice/crio-406dfcfa48254ee09241b6a6102dc1873ab49684c6a47f19d29305abb88c05c3 WatchSource:0}: Error finding container 406dfcfa48254ee09241b6a6102dc1873ab49684c6a47f19d29305abb88c05c3: Status 404 returned error can't find the container with id 406dfcfa48254ee09241b6a6102dc1873ab49684c6a47f19d29305abb88c05c3
Sep 11 17:45:36.454951 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.454921    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:36.555585 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.555552    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:36.757403 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:36.757374    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq" event={"ID":"0ce262a3-df86-49d9-ab72-6eab8d280c04","Type":"ContainerStarted","Data":"406dfcfa48254ee09241b6a6102dc1873ab49684c6a47f19d29305abb88c05c3"}
Sep 11 17:45:37.524886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:37.524853    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:37.629016 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:37.628956    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:38.054153 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:38.054106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:38.165759 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:38.165723    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:38.768277 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:38.768242    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq" event={"ID":"0ce262a3-df86-49d9-ab72-6eab8d280c04","Type":"ContainerStarted","Data":"32f8ebe9a909910c46d0905fb53c61f3e3a0ab94685d27c9bfeda8e2b9633dd4"}
Sep 11 17:45:38.768727 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:38.768491    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:38.788778 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:38.788731    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq" podStartSLOduration=1.938927197 podStartE2EDuration="3.788717634s" totalImagesPullingTime="1.849790437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:38.788340732 +0000 UTC m=+421.658383688" watchObservedRunningTime="2026-09-11 17:45:38.788717634 +0000 UTC m=+421.658760579"
Sep 11 17:45:40.379804 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.379768    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"]
Sep 11 17:45:40.383128 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.383111    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.385791 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.385769    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jcg7f\""
Sep 11 17:45:40.400477 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.400450    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"]
Sep 11 17:45:40.555652 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.555619    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2frqg\" (UniqueName: \"kubernetes.io/projected/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-kube-api-access-2frqg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.555812 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.555669    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.656698 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.656629    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.656789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.656705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2frqg\" (UniqueName: \"kubernetes.io/projected/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-kube-api-access-2frqg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.657013 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.656992    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.666147 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.666115    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2frqg\" (UniqueName: \"kubernetes.io/projected/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-kube-api-access-2frqg\") pod \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.692917 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.692888    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:40.963183 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:40.963140    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"]
Sep 11 17:45:40.965954 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:40.965925    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd39c848d_79e9_40ed_8f1b_576d1c7c39e8.slice/crio-c90d274e44d1d30e10a84a30b3f17f424f7cbb638fe9a6e86403301ddc60302b WatchSource:0}: Error finding container c90d274e44d1d30e10a84a30b3f17f424f7cbb638fe9a6e86403301ddc60302b: Status 404 returned error can't find the container with id c90d274e44d1d30e10a84a30b3f17f424f7cbb638fe9a6e86403301ddc60302b
Sep 11 17:45:41.685692 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:41.685662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:41.784238 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:41.784194    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b" event={"ID":"d39c848d-79e9-40ed-8f1b-576d1c7c39e8","Type":"ContainerStarted","Data":"c90d274e44d1d30e10a84a30b3f17f424f7cbb638fe9a6e86403301ddc60302b"}
Sep 11 17:45:41.785082 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:41.785050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:44.802795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:44.802760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b" event={"ID":"d39c848d-79e9-40ed-8f1b-576d1c7c39e8","Type":"ContainerStarted","Data":"2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb"}
Sep 11 17:45:44.803220 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:44.802887    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:44.827608 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:44.827566    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b" podStartSLOduration=1.138467224 podStartE2EDuration="4.827550988s" totalImagesPullingTime="3.689083764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:44.826091476 +0000 UTC m=+427.696134420" watchObservedRunningTime="2026-09-11 17:45:44.827550988 +0000 UTC m=+427.697593933"
Sep 11 17:45:44.846324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:44.846292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:44.948750 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:44.948677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:46.760559 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:46.760531    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-wzvc8"
Sep 11 17:45:47.488770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.488731    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"]
Sep 11 17:45:47.491949 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.491922    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:47.492217 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.492203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.494433 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.494411    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 17:45:47.494539 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.494432    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 17:45:47.494539 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.494450    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8mbx2\""
Sep 11 17:45:47.502498 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.502474    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"]
Sep 11 17:45:47.514979 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.514957    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32a132e8-ee3c-42f4-bd77-fd2889af3040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.515077 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.515028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckxrt\" (UniqueName: \"kubernetes.io/projected/32a132e8-ee3c-42f4-bd77-fd2889af3040-kube-api-access-ckxrt\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.515077 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.515066    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/32a132e8-ee3c-42f4-bd77-fd2889af3040-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.616176 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.616148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32a132e8-ee3c-42f4-bd77-fd2889af3040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.616286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.616219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckxrt\" (UniqueName: \"kubernetes.io/projected/32a132e8-ee3c-42f4-bd77-fd2889af3040-kube-api-access-ckxrt\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.616286 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.616250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/32a132e8-ee3c-42f4-bd77-fd2889af3040-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.616405 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:47.616291    2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 11 17:45:47.616405 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:47.616379    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32a132e8-ee3c-42f4-bd77-fd2889af3040-plugin-serving-cert podName:32a132e8-ee3c-42f4-bd77-fd2889af3040 nodeName:}" failed. No retries permitted until 2026-09-11 17:45:48.11633196 +0000 UTC m=+430.986374891 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/32a132e8-ee3c-42f4-bd77-fd2889af3040-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-kn68h" (UID: "32a132e8-ee3c-42f4-bd77-fd2889af3040") : secret "plugin-serving-cert" not found
Sep 11 17:45:47.616909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.616891    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/32a132e8-ee3c-42f4-bd77-fd2889af3040-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.626358 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.626329    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckxrt\" (UniqueName: \"kubernetes.io/projected/32a132e8-ee3c-42f4-bd77-fd2889af3040-kube-api-access-ckxrt\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:47.760658 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.760578    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:47.820121 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.820091    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc" event={"ID":"d1d5f3d1-1641-44ba-a8c6-470a28043efa","Type":"ContainerStarted","Data":"252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a"}
Sep 11 17:45:47.820262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.820153    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:47.840005 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:47.839958    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc" podStartSLOduration=1.5463953940000001 podStartE2EDuration="14.839942315s" totalImagesPullingTime="13.293546921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:47.838101614 +0000 UTC m=+430.708144559" watchObservedRunningTime="2026-09-11 17:45:47.839942315 +0000 UTC m=+430.709985253"
Sep 11 17:45:48.119805 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:48.119719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32a132e8-ee3c-42f4-bd77-fd2889af3040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:48.122059 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:48.122033    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/32a132e8-ee3c-42f4-bd77-fd2889af3040-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-kn68h\" (UID: \"32a132e8-ee3c-42f4-bd77-fd2889af3040\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:48.404020 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:48.403930    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"
Sep 11 17:45:48.673238 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:48.673206    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h"]
Sep 11 17:45:48.675315 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:48.675288    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32a132e8_ee3c_42f4_bd77_fd2889af3040.slice/crio-972e4b9b96b5cefe06d799a3b38d4e9ab35c923d9033d683c8903be7070ccc78 WatchSource:0}: Error finding container 972e4b9b96b5cefe06d799a3b38d4e9ab35c923d9033d683c8903be7070ccc78: Status 404 returned error can't find the container with id 972e4b9b96b5cefe06d799a3b38d4e9ab35c923d9033d683c8903be7070ccc78
Sep 11 17:45:48.825142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:48.825106    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h" event={"ID":"32a132e8-ee3c-42f4-bd77-fd2889af3040","Type":"ContainerStarted","Data":"972e4b9b96b5cefe06d799a3b38d4e9ab35c923d9033d683c8903be7070ccc78"}
Sep 11 17:45:49.306688 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:49.306647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:49.406937 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:49.406895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:49.774998 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:49.774973    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-gbxbq"
Sep 11 17:45:54.088689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:54.088653    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:54.185785 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:54.185743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:54.289466 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:54.289429    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:45:55.809859 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:55.809822    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:56.872135 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:56.872097    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h" event={"ID":"32a132e8-ee3c-42f4-bd77-fd2889af3040","Type":"ContainerStarted","Data":"c0f867b8e4a23e0a13efb549ec2e7250dfbab2442b3f05fdf2a97f7068417b33"}
Sep 11 17:45:56.946646 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:56.946583    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-kn68h" podStartSLOduration=2.410947383 podStartE2EDuration="9.94656781s" totalImagesPullingTime="7.535620427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:56.945417609 +0000 UTC m=+439.815460554" watchObservedRunningTime="2026-09-11 17:45:56.94656781 +0000 UTC m=+439.816610773"
Sep 11 17:45:57.623262 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.623220    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"]
Sep 11 17:45:57.626963 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.626940    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.649926 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.649895    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"]
Sep 11 17:45:57.701938 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.701908    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpkcl\" (UniqueName: \"kubernetes.io/projected/a806fd7f-d77b-4a42-aede-65223ff86748-kube-api-access-dpkcl\") pod \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.702072 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.701970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a806fd7f-d77b-4a42-aede-65223ff86748-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.803318 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.803279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpkcl\" (UniqueName: \"kubernetes.io/projected/a806fd7f-d77b-4a42-aede-65223ff86748-kube-api-access-dpkcl\") pod \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.803477 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.803388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a806fd7f-d77b-4a42-aede-65223ff86748-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.803746 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.803728    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a806fd7f-d77b-4a42-aede-65223ff86748-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.819486 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.819462    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpkcl\" (UniqueName: \"kubernetes.io/projected/a806fd7f-d77b-4a42-aede-65223ff86748-kube-api-access-dpkcl\") pod \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:57.936752 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:57.936689    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:58.217112 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.217070    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"]
Sep 11 17:45:58.219528 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:58.219497    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda806fd7f_d77b_4a42_aede_65223ff86748.slice/crio-9dbba2dc05455639d22e173eff672f44bb03e1784d9770420e4a59ac2b0f3268 WatchSource:0}: Error finding container 9dbba2dc05455639d22e173eff672f44bb03e1784d9770420e4a59ac2b0f3268: Status 404 returned error can't find the container with id 9dbba2dc05455639d22e173eff672f44bb03e1784d9770420e4a59ac2b0f3268
Sep 11 17:45:58.279001 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.278972    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"]
Sep 11 17:45:58.283466 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.283441    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"]
Sep 11 17:45:58.298293 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.298269    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"]
Sep 11 17:45:58.298499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.298472    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" containerName="manager" containerID="cri-o://2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb" gracePeriod=2
Sep 11 17:45:58.313375 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.313334    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"]
Sep 11 17:45:58.313686 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.313674    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" containerName="manager"
Sep 11 17:45:58.313727 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.313688    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" containerName="manager"
Sep 11 17:45:58.313764 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.313752    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" containerName="manager"
Sep 11 17:45:58.316643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.316627    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.325364 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.325331    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"]
Sep 11 17:45:58.328428 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.328402    2582 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\": pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.332050 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.332020    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.335931 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.335906    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"]
Sep 11 17:45:58.337772 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.337749    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"]
Sep 11 17:45:58.337977 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.337952    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" containerName="manager" containerID="cri-o://252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a" gracePeriod=2
Sep 11 17:45:58.339487 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.339468    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.345163 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.345142    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"]
Sep 11 17:45:58.345482 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.345469    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" containerName="manager"
Sep 11 17:45:58.345541 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.345484    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" containerName="manager"
Sep 11 17:45:58.345624 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.345553    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" containerName="manager"
Sep 11 17:45:58.348816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.348713    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.351949 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.351928    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"]
Sep 11 17:45:58.356539 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.356513    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.358270 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.358240    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.377424 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.377399    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"]
Sep 11 17:45:58.407263 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.407235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qnzm\" (UniqueName: \"kubernetes.io/projected/188c8e60-81b6-46f1-afa6-d49053398b97-kube-api-access-7qnzm\") pod \"kuadrant-operator-controller-manager-5f685c496f-zs2b2\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.407424 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.407303    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6prj\" (UniqueName: \"kubernetes.io/projected/8439b2e4-7136-42ff-968a-1a74b35bc018-kube-api-access-m6prj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jx765\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.407424 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.407393    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8439b2e4-7136-42ff-968a-1a74b35bc018-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jx765\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.407511 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.407454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/188c8e60-81b6-46f1-afa6-d49053398b97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-zs2b2\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.508935 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.508899    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8439b2e4-7136-42ff-968a-1a74b35bc018-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jx765\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.509063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.508972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/188c8e60-81b6-46f1-afa6-d49053398b97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-zs2b2\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.509063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.509023    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qnzm\" (UniqueName: \"kubernetes.io/projected/188c8e60-81b6-46f1-afa6-d49053398b97-kube-api-access-7qnzm\") pod \"kuadrant-operator-controller-manager-5f685c496f-zs2b2\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.509192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.509112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6prj\" (UniqueName: \"kubernetes.io/projected/8439b2e4-7136-42ff-968a-1a74b35bc018-kube-api-access-m6prj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jx765\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.509331 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.509306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8439b2e4-7136-42ff-968a-1a74b35bc018-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jx765\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.509484 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.509385    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/188c8e60-81b6-46f1-afa6-d49053398b97-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-zs2b2\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.517819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.517792    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6prj\" (UniqueName: \"kubernetes.io/projected/8439b2e4-7136-42ff-968a-1a74b35bc018-kube-api-access-m6prj\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jx765\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.517928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.517911    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qnzm\" (UniqueName: \"kubernetes.io/projected/188c8e60-81b6-46f1-afa6-d49053398b97-kube-api-access-7qnzm\") pod \"kuadrant-operator-controller-manager-5f685c496f-zs2b2\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.563212 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.563190    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.565117 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.565091    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.566650 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.566630    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.567093 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.567078    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.568589 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.568567    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.570169 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.570138    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.609789 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.609765    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-extensions-socket-volume\") pod \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") "
Sep 11 17:45:58.609868 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.609811    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8jzx\" (UniqueName: \"kubernetes.io/projected/d1d5f3d1-1641-44ba-a8c6-470a28043efa-kube-api-access-x8jzx\") pod \"d1d5f3d1-1641-44ba-a8c6-470a28043efa\" (UID: \"d1d5f3d1-1641-44ba-a8c6-470a28043efa\") "
Sep 11 17:45:58.609868 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.609835    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2frqg\" (UniqueName: \"kubernetes.io/projected/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-kube-api-access-2frqg\") pod \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\" (UID: \"d39c848d-79e9-40ed-8f1b-576d1c7c39e8\") "
Sep 11 17:45:58.610266 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.610242    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d39c848d-79e9-40ed-8f1b-576d1c7c39e8" (UID: "d39c848d-79e9-40ed-8f1b-576d1c7c39e8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:58.611891 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.611868    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d5f3d1-1641-44ba-a8c6-470a28043efa-kube-api-access-x8jzx" (OuterVolumeSpecName: "kube-api-access-x8jzx") pod "d1d5f3d1-1641-44ba-a8c6-470a28043efa" (UID: "d1d5f3d1-1641-44ba-a8c6-470a28043efa"). InnerVolumeSpecName "kube-api-access-x8jzx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:58.611967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.611899    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-kube-api-access-2frqg" (OuterVolumeSpecName: "kube-api-access-2frqg") pod "d39c848d-79e9-40ed-8f1b-576d1c7c39e8" (UID: "d39c848d-79e9-40ed-8f1b-576d1c7c39e8"). InnerVolumeSpecName "kube-api-access-2frqg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:58.710501 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.710466    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:45:58.710501 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.710495    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8jzx\" (UniqueName: \"kubernetes.io/projected/d1d5f3d1-1641-44ba-a8c6-470a28043efa-kube-api-access-x8jzx\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:45:58.710501 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.710503    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2frqg\" (UniqueName: \"kubernetes.io/projected/d39c848d-79e9-40ed-8f1b-576d1c7c39e8-kube-api-access-2frqg\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:45:58.743309 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.743279    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:58.751277 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.751211    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:58.883244 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.883205    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" containerName="manager" containerID="cri-o://755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49" gracePeriod=2
Sep 11 17:45:58.884646 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.884615    2582 generic.go:356] "Generic (PLEG): container finished" podID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" containerID="252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a" exitCode=0
Sep 11 17:45:58.884770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.884652    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.884770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.884712    2582 scope.go:117] "RemoveContainer" containerID="252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a"
Sep 11 17:45:58.885236 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.885197    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:58.886468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.886445    2582 generic.go:356] "Generic (PLEG): container finished" podID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" containerID="2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb" exitCode=0
Sep 11 17:45:58.886572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.886482    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.887043 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.887011    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.888659 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.888615    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.890598 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.890560    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.892306 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.892276    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:58.893943 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.893898    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.894264 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.894242    2582 scope.go:117] "RemoveContainer" containerID="252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a"
Sep 11 17:45:58.894566 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.894546    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a\": container with ID starting with 252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a not found: ID does not exist" containerID="252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a"
Sep 11 17:45:58.894618 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.894575    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a"} err="failed to get container status \"252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a\": rpc error: code = NotFound desc = could not find container \"252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a\": container with ID starting with 252fd234fd49c3c0d593c21bb9a866c943d09e9620e6243f1baccf43aa711a1a not found: ID does not exist"
Sep 11 17:45:58.894618 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.894592    2582 scope.go:117] "RemoveContainer" containerID="2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb"
Sep 11 17:45:58.953868 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.953839    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.955498 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.955478    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.956933 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.956912    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:58.958691 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.958673    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:58.960233 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.960210    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:58.961931 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.961912    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:58.998227 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.998063    2582 scope.go:117] "RemoveContainer" containerID="2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb"
Sep 11 17:45:58.998466 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:58.998431    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb\": container with ID starting with 2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb not found: ID does not exist" containerID="2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb"
Sep 11 17:45:58.998561 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:58.998475    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb"} err="failed to get container status \"2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb\": rpc error: code = NotFound desc = could not find container \"2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb\": container with ID starting with 2456390413dd0eed536b73f7fb41e01e835cae36a981a7698e07001542a9e1bb not found: ID does not exist"
Sep 11 17:45:59.020915 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.020884    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"]
Sep 11 17:45:59.024285 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:59.024237    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188c8e60_81b6_46f1_afa6_d49053398b97.slice/crio-9fba1ff127c75855e442137a4a562e1fb1071c3b535a49826d0865363c8c357a WatchSource:0}: Error finding container 9fba1ff127c75855e442137a4a562e1fb1071c3b535a49826d0865363c8c357a: Status 404 returned error can't find the container with id 9fba1ff127c75855e442137a4a562e1fb1071c3b535a49826d0865363c8c357a
Sep 11 17:45:59.025467 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.025442    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"]
Sep 11 17:45:59.027830 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:45:59.027802    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8439b2e4_7136_42ff_968a_1a74b35bc018.slice/crio-cdd6e76a11e07742bde5abe68fd6a35584ff045b20b177eab2e7bbf1d899f408 WatchSource:0}: Error finding container cdd6e76a11e07742bde5abe68fd6a35584ff045b20b177eab2e7bbf1d899f408: Status 404 returned error can't find the container with id cdd6e76a11e07742bde5abe68fd6a35584ff045b20b177eab2e7bbf1d899f408
Sep 11 17:45:59.122898 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.122872    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:59.124859 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:59.124838    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-2vddc\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-2vddc"
Sep 11 17:45:59.126719 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:59.126697    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:59.128474 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:59.128454    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-xpt4b\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-xpt4b"
Sep 11 17:45:59.215494 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.215464    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a806fd7f-d77b-4a42-aede-65223ff86748-extensions-socket-volume\") pod \"a806fd7f-d77b-4a42-aede-65223ff86748\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") "
Sep 11 17:45:59.215639 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.215513    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dpkcl\" (UniqueName: \"kubernetes.io/projected/a806fd7f-d77b-4a42-aede-65223ff86748-kube-api-access-dpkcl\") pod \"a806fd7f-d77b-4a42-aede-65223ff86748\" (UID: \"a806fd7f-d77b-4a42-aede-65223ff86748\") "
Sep 11 17:45:59.215726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.215702    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a806fd7f-d77b-4a42-aede-65223ff86748-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a806fd7f-d77b-4a42-aede-65223ff86748" (UID: "a806fd7f-d77b-4a42-aede-65223ff86748"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:45:59.217523 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.217502    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a806fd7f-d77b-4a42-aede-65223ff86748-kube-api-access-dpkcl" (OuterVolumeSpecName: "kube-api-access-dpkcl") pod "a806fd7f-d77b-4a42-aede-65223ff86748" (UID: "a806fd7f-d77b-4a42-aede-65223ff86748"). InnerVolumeSpecName "kube-api-access-dpkcl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:45:59.316091 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.316013    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a806fd7f-d77b-4a42-aede-65223ff86748-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:45:59.316091 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.316041    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpkcl\" (UniqueName: \"kubernetes.io/projected/a806fd7f-d77b-4a42-aede-65223ff86748-kube-api-access-dpkcl\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:45:59.747221 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.747189    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" path="/var/lib/kubelet/pods/a806fd7f-d77b-4a42-aede-65223ff86748/volumes"
Sep 11 17:45:59.747620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.747604    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" path="/var/lib/kubelet/pods/d1d5f3d1-1641-44ba-a8c6-470a28043efa/volumes"
Sep 11 17:45:59.747947 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.747935    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d39c848d-79e9-40ed-8f1b-576d1c7c39e8" path="/var/lib/kubelet/pods/d39c848d-79e9-40ed-8f1b-576d1c7c39e8/volumes"
Sep 11 17:45:59.892293 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.892254    2582 generic.go:356] "Generic (PLEG): container finished" podID="a806fd7f-d77b-4a42-aede-65223ff86748" containerID="755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49" exitCode=2
Sep 11 17:45:59.892592 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.892302    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:59.892592 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.892321    2582 scope.go:117] "RemoveContainer" containerID="755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49"
Sep 11 17:45:59.896476 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.896438    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765" event={"ID":"8439b2e4-7136-42ff-968a-1a74b35bc018","Type":"ContainerStarted","Data":"08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d"}
Sep 11 17:45:59.896590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.896499    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765" event={"ID":"8439b2e4-7136-42ff-968a-1a74b35bc018","Type":"ContainerStarted","Data":"cdd6e76a11e07742bde5abe68fd6a35584ff045b20b177eab2e7bbf1d899f408"}
Sep 11 17:45:59.896590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.896532    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:45:59.898153 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.898131    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2" event={"ID":"188c8e60-81b6-46f1-afa6-d49053398b97","Type":"ContainerStarted","Data":"fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223"}
Sep 11 17:45:59.898264 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.898166    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2" event={"ID":"188c8e60-81b6-46f1-afa6-d49053398b97","Type":"ContainerStarted","Data":"9fba1ff127c75855e442137a4a562e1fb1071c3b535a49826d0865363c8c357a"}
Sep 11 17:45:59.898264 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.898190    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:45:59.899224 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:59.899201    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:59.901085 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.901069    2582 scope.go:117] "RemoveContainer" containerID="755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49"
Sep 11 17:45:59.901340 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:59.901320    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49\": container with ID starting with 755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49 not found: ID does not exist" containerID="755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49"
Sep 11 17:45:59.901482 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.901358    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49"} err="failed to get container status \"755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49\": rpc error: code = NotFound desc = could not find container \"755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49\": container with ID starting with 755cd5719edd215fbe4d4f70938fc4439b781a357560195f46a092541c17bf49 not found: ID does not exist"
Sep 11 17:45:59.932482 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.932435    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2" podStartSLOduration=1.932419307 podStartE2EDuration="1.932419307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:59.930400356 +0000 UTC m=+442.800443298" watchObservedRunningTime="2026-09-11 17:45:59.932419307 +0000 UTC m=+442.802462251"
Sep 11 17:45:59.933437 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:45:59.933415    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-24w4q\" is forbidden: User \"system:node:ip-10-0-128-237.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-237.ec2.internal' and this object" podUID="a806fd7f-d77b-4a42-aede-65223ff86748" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-24w4q"
Sep 11 17:45:59.988306 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:45:59.988261    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765" podStartSLOduration=1.988244135 podStartE2EDuration="1.988244135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:45:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:45:59.986897966 +0000 UTC m=+442.856940910" watchObservedRunningTime="2026-09-11 17:45:59.988244135 +0000 UTC m=+442.858287080"
Sep 11 17:46:00.795870 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:00.795838    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:00.908530 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:00.908500    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:03.964374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:03.964321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:04.065117 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:04.065071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:08.417821 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:08.417767    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:08.481910 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:08.481882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:08.551078 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:08.551023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:08.948294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:08.948258    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:10.906536 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:10.906504    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:46:10.907005 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:10.906577    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:46:10.991044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:10.991012    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"]
Sep 11 17:46:10.991255 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:10.991208    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765" podUID="8439b2e4-7136-42ff-968a-1a74b35bc018" containerName="manager" containerID="cri-o://08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d" gracePeriod=10
Sep 11 17:46:11.241015 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.240986    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:46:11.307790 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.307764    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8439b2e4-7136-42ff-968a-1a74b35bc018-extensions-socket-volume\") pod \"8439b2e4-7136-42ff-968a-1a74b35bc018\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") "
Sep 11 17:46:11.307936 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.307807    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m6prj\" (UniqueName: \"kubernetes.io/projected/8439b2e4-7136-42ff-968a-1a74b35bc018-kube-api-access-m6prj\") pod \"8439b2e4-7136-42ff-968a-1a74b35bc018\" (UID: \"8439b2e4-7136-42ff-968a-1a74b35bc018\") "
Sep 11 17:46:11.308107 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.308084    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8439b2e4-7136-42ff-968a-1a74b35bc018-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "8439b2e4-7136-42ff-968a-1a74b35bc018" (UID: "8439b2e4-7136-42ff-968a-1a74b35bc018"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:46:11.309730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.309707    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8439b2e4-7136-42ff-968a-1a74b35bc018-kube-api-access-m6prj" (OuterVolumeSpecName: "kube-api-access-m6prj") pod "8439b2e4-7136-42ff-968a-1a74b35bc018" (UID: "8439b2e4-7136-42ff-968a-1a74b35bc018"). InnerVolumeSpecName "kube-api-access-m6prj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:46:11.329513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329491    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"]
Sep 11 17:46:11.329827 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329815    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8439b2e4-7136-42ff-968a-1a74b35bc018" containerName="manager"
Sep 11 17:46:11.329873 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329828    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8439b2e4-7136-42ff-968a-1a74b35bc018" containerName="manager"
Sep 11 17:46:11.329873 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329844    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" containerName="manager"
Sep 11 17:46:11.329873 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329850    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" containerName="manager"
Sep 11 17:46:11.329962 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329904    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1d5f3d1-1641-44ba-a8c6-470a28043efa" containerName="manager"
Sep 11 17:46:11.329962 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.329912    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8439b2e4-7136-42ff-968a-1a74b35bc018" containerName="manager"
Sep 11 17:46:11.333249 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.333225    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.347200 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.347179    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"]
Sep 11 17:46:11.408896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.408833    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7vl8x\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.408896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.408872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdssb\" (UniqueName: \"kubernetes.io/projected/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-kube-api-access-mdssb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7vl8x\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.409036 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.408946    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8439b2e4-7136-42ff-968a-1a74b35bc018-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:46:11.409036 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.408964    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m6prj\" (UniqueName: \"kubernetes.io/projected/8439b2e4-7136-42ff-968a-1a74b35bc018-kube-api-access-m6prj\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:46:11.509838 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.509807    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7vl8x\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.509980 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.509855    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdssb\" (UniqueName: \"kubernetes.io/projected/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-kube-api-access-mdssb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7vl8x\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.510132 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.510112    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7vl8x\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.528469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.528434    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdssb\" (UniqueName: \"kubernetes.io/projected/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-kube-api-access-mdssb\") pod \"kuadrant-operator-controller-manager-5769f99d9f-7vl8x\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.643377 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.643332    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:11.919706 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.919667    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"]
Sep 11 17:46:11.927637 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:46:11.927612    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89d4eb8d_7a76_445f_beac_d9cc2a92c3c1.slice/crio-1db7800356da85c6837800f056c320e662a345e540d6eb005a9c12e3c5f27d69 WatchSource:0}: Error finding container 1db7800356da85c6837800f056c320e662a345e540d6eb005a9c12e3c5f27d69: Status 404 returned error can't find the container with id 1db7800356da85c6837800f056c320e662a345e540d6eb005a9c12e3c5f27d69
Sep 11 17:46:11.957093 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.957063    2582 generic.go:356] "Generic (PLEG): container finished" podID="8439b2e4-7136-42ff-968a-1a74b35bc018" containerID="08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d" exitCode=0
Sep 11 17:46:11.957197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.957112    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"
Sep 11 17:46:11.957197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.957133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765" event={"ID":"8439b2e4-7136-42ff-968a-1a74b35bc018","Type":"ContainerDied","Data":"08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d"}
Sep 11 17:46:11.957197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.957174    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765" event={"ID":"8439b2e4-7136-42ff-968a-1a74b35bc018","Type":"ContainerDied","Data":"cdd6e76a11e07742bde5abe68fd6a35584ff045b20b177eab2e7bbf1d899f408"}
Sep 11 17:46:11.957197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.957194    2582 scope.go:117] "RemoveContainer" containerID="08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d"
Sep 11 17:46:11.958477 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.958459    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x" event={"ID":"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1","Type":"ContainerStarted","Data":"1db7800356da85c6837800f056c320e662a345e540d6eb005a9c12e3c5f27d69"}
Sep 11 17:46:11.965498 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.965471    2582 scope.go:117] "RemoveContainer" containerID="08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d"
Sep 11 17:46:11.965800 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:46:11.965778    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d\": container with ID starting with 08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d not found: ID does not exist" containerID="08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d"
Sep 11 17:46:11.965874 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.965808    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d"} err="failed to get container status \"08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d\": rpc error: code = NotFound desc = could not find container \"08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d\": container with ID starting with 08b9490cbb755767b0306fa2c19443f1dcaa8b1f0e39f5f860b459ae467a084d not found: ID does not exist"
Sep 11 17:46:11.981479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.981456    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"]
Sep 11 17:46:11.985540 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:11.985519    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jx765"]
Sep 11 17:46:12.965470 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:12.965433    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x" event={"ID":"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1","Type":"ContainerStarted","Data":"77e41d4ce950e21d8727680b69de3c4a54b149d21040efeca7cde235793bce66"}
Sep 11 17:46:12.965808 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:12.965560    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:12.987157 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:12.987118    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x" podStartSLOduration=1.9871053939999999 podStartE2EDuration="1.987105394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:12.986449476 +0000 UTC m=+455.856492420" watchObservedRunningTime="2026-09-11 17:46:12.987105394 +0000 UTC m=+455.857148320"
Sep 11 17:46:13.619211 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:13.619173    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:13.713233 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:13.713196    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:13.746031 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:13.746004    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8439b2e4-7136-42ff-968a-1a74b35bc018" path="/var/lib/kubelet/pods/8439b2e4-7136-42ff-968a-1a74b35bc018/volumes"
Sep 11 17:46:13.817556 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:13.817523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:23.972488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:23.972456    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 17:46:24.029308 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.029275    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"]
Sep 11 17:46:24.029562 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.029520    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2" podUID="188c8e60-81b6-46f1-afa6-d49053398b97" containerName="manager" containerID="cri-o://fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223" gracePeriod=10
Sep 11 17:46:24.273376 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.273340    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:46:24.412714 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.412681    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7qnzm\" (UniqueName: \"kubernetes.io/projected/188c8e60-81b6-46f1-afa6-d49053398b97-kube-api-access-7qnzm\") pod \"188c8e60-81b6-46f1-afa6-d49053398b97\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") "
Sep 11 17:46:24.412873 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.412742    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/188c8e60-81b6-46f1-afa6-d49053398b97-extensions-socket-volume\") pod \"188c8e60-81b6-46f1-afa6-d49053398b97\" (UID: \"188c8e60-81b6-46f1-afa6-d49053398b97\") "
Sep 11 17:46:24.413128 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.413101    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/188c8e60-81b6-46f1-afa6-d49053398b97-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "188c8e60-81b6-46f1-afa6-d49053398b97" (UID: "188c8e60-81b6-46f1-afa6-d49053398b97"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:46:24.414803 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.414771    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/188c8e60-81b6-46f1-afa6-d49053398b97-kube-api-access-7qnzm" (OuterVolumeSpecName: "kube-api-access-7qnzm") pod "188c8e60-81b6-46f1-afa6-d49053398b97" (UID: "188c8e60-81b6-46f1-afa6-d49053398b97"). InnerVolumeSpecName "kube-api-access-7qnzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:46:24.513288 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.513200    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/188c8e60-81b6-46f1-afa6-d49053398b97-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:46:24.513288 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:24.513227    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7qnzm\" (UniqueName: \"kubernetes.io/projected/188c8e60-81b6-46f1-afa6-d49053398b97-kube-api-access-7qnzm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:46:25.027192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.027158    2582 generic.go:356] "Generic (PLEG): container finished" podID="188c8e60-81b6-46f1-afa6-d49053398b97" containerID="fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223" exitCode=0
Sep 11 17:46:25.027715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.027242    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"
Sep 11 17:46:25.027715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.027239    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2" event={"ID":"188c8e60-81b6-46f1-afa6-d49053398b97","Type":"ContainerDied","Data":"fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223"}
Sep 11 17:46:25.027715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.027441    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2" event={"ID":"188c8e60-81b6-46f1-afa6-d49053398b97","Type":"ContainerDied","Data":"9fba1ff127c75855e442137a4a562e1fb1071c3b535a49826d0865363c8c357a"}
Sep 11 17:46:25.027715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.027470    2582 scope.go:117] "RemoveContainer" containerID="fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223"
Sep 11 17:46:25.036920 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.036729    2582 scope.go:117] "RemoveContainer" containerID="fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223"
Sep 11 17:46:25.037016 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:46:25.036997    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223\": container with ID starting with fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223 not found: ID does not exist" containerID="fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223"
Sep 11 17:46:25.037061 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.037023    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223"} err="failed to get container status \"fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223\": rpc error: code = NotFound desc = could not find container \"fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223\": container with ID starting with fae8b3f494fd409d4bff5449ea0029607f2663e85c9b67e93ba1db34f901b223 not found: ID does not exist"
Sep 11 17:46:25.052962 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.052936    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"]
Sep 11 17:46:25.064193 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.064165    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-zs2b2"]
Sep 11 17:46:25.746116 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:25.746083    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="188c8e60-81b6-46f1-afa6-d49053398b97" path="/var/lib/kubelet/pods/188c8e60-81b6-46f1-afa6-d49053398b97/volumes"
Sep 11 17:46:27.620476 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:27.620425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:27.718906 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:27.718859    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:36.427981 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:36.427940    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:40.591167 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.591130    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:40.591583 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.591487    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188c8e60-81b6-46f1-afa6-d49053398b97" containerName="manager"
Sep 11 17:46:40.591583 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.591499    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="188c8e60-81b6-46f1-afa6-d49053398b97" containerName="manager"
Sep 11 17:46:40.591583 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.591575    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="188c8e60-81b6-46f1-afa6-d49053398b97" containerName="manager"
Sep 11 17:46:40.594563 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.594542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.595780 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.595758    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:40.596812 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.596794    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 17:46:40.607229 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.607205    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:40.688112 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.688077    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:40.707161 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.707129    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:40.739270 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.739240    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-config-file\") pod \"limitador-limitador-8588749977-jff2m\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") " pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.739438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.739281    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-546ht\" (UniqueName: \"kubernetes.io/projected/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-kube-api-access-546ht\") pod \"limitador-limitador-8588749977-jff2m\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") " pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.767714 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.767685    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:40.840695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.840664    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-config-file\") pod \"limitador-limitador-8588749977-jff2m\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") " pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.840853 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.840705    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-546ht\" (UniqueName: \"kubernetes.io/projected/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-kube-api-access-546ht\") pod \"limitador-limitador-8588749977-jff2m\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") " pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.841367 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.841308    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-config-file\") pod \"limitador-limitador-8588749977-jff2m\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") " pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.848172 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.848148    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-546ht\" (UniqueName: \"kubernetes.io/projected/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-kube-api-access-546ht\") pod \"limitador-limitador-8588749977-jff2m\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") " pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.904841 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.904813    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:40.908316 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.908294    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:46:40.911743 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.911725    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:40.919795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.919771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:46:40.949180 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:40.949142    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:46:41.042089 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.042055    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ada2023-141e-4e73-abfd-8abc6c4b9169-config-file\") pod \"limitador-limitador-84994f7ffc-bg4v4\" (UID: \"2ada2023-141e-4e73-abfd-8abc6c4b9169\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.042242 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.042114    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg4hf\" (UniqueName: \"kubernetes.io/projected/2ada2023-141e-4e73-abfd-8abc6c4b9169-kube-api-access-wg4hf\") pod \"limitador-limitador-84994f7ffc-bg4v4\" (UID: \"2ada2023-141e-4e73-abfd-8abc6c4b9169\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.142924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.142856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ada2023-141e-4e73-abfd-8abc6c4b9169-config-file\") pod \"limitador-limitador-84994f7ffc-bg4v4\" (UID: \"2ada2023-141e-4e73-abfd-8abc6c4b9169\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.142924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.142903    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg4hf\" (UniqueName: \"kubernetes.io/projected/2ada2023-141e-4e73-abfd-8abc6c4b9169-kube-api-access-wg4hf\") pod \"limitador-limitador-84994f7ffc-bg4v4\" (UID: \"2ada2023-141e-4e73-abfd-8abc6c4b9169\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.143537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.143515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2ada2023-141e-4e73-abfd-8abc6c4b9169-config-file\") pod \"limitador-limitador-84994f7ffc-bg4v4\" (UID: \"2ada2023-141e-4e73-abfd-8abc6c4b9169\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.151671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.151650    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg4hf\" (UniqueName: \"kubernetes.io/projected/2ada2023-141e-4e73-abfd-8abc6c4b9169-kube-api-access-wg4hf\") pod \"limitador-limitador-84994f7ffc-bg4v4\" (UID: \"2ada2023-141e-4e73-abfd-8abc6c4b9169\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.175184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.175082    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:41.177448 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:46:41.177413    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02dc0fe9_3f60_4f3b_a5a6_5811bf543f3e.slice/crio-5d6edd75b816a61cacec167de148d0e74d3586fb814524e79c0e35a21c4584f4 WatchSource:0}: Error finding container 5d6edd75b816a61cacec167de148d0e74d3586fb814524e79c0e35a21c4584f4: Status 404 returned error can't find the container with id 5d6edd75b816a61cacec167de148d0e74d3586fb814524e79c0e35a21c4584f4
Sep 11 17:46:41.244117 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.244088    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:41.429012 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.428923    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:41.515681 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.515651    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:46:41.517438 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:46:41.517403    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ada2023_141e_4e73_abfd_8abc6c4b9169.slice/crio-923b514212363bbeb43958c62dfc5b2df3cdddff7e82fb607e9406f5c5341608 WatchSource:0}: Error finding container 923b514212363bbeb43958c62dfc5b2df3cdddff7e82fb607e9406f5c5341608: Status 404 returned error can't find the container with id 923b514212363bbeb43958c62dfc5b2df3cdddff7e82fb607e9406f5c5341608
Sep 11 17:46:41.517611 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:41.517489    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:42.118408 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:42.118373    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4" event={"ID":"2ada2023-141e-4e73-abfd-8abc6c4b9169","Type":"ContainerStarted","Data":"923b514212363bbeb43958c62dfc5b2df3cdddff7e82fb607e9406f5c5341608"}
Sep 11 17:46:42.119919 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:42.119884    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jff2m" event={"ID":"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e","Type":"ContainerStarted","Data":"5d6edd75b816a61cacec167de148d0e74d3586fb814524e79c0e35a21c4584f4"}
Sep 11 17:46:43.125832 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.125794    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4" event={"ID":"2ada2023-141e-4e73-abfd-8abc6c4b9169","Type":"ContainerStarted","Data":"76f348a3b76c6a6d7fb1383067c044bc633518cbd0981fcc565b7328344552db"}
Sep 11 17:46:43.126268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.125894    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:43.127419 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.127401    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jff2m" event={"ID":"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e","Type":"ContainerStarted","Data":"c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e"}
Sep 11 17:46:43.127527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.127513    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:43.142989 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.142945    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4" podStartSLOduration=2.247653222 podStartE2EDuration="3.142929057s" totalImagesPullingTime="895.275835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:43.141569306 +0000 UTC m=+486.011612251" watchObservedRunningTime="2026-09-11 17:46:43.142929057 +0000 UTC m=+486.012972001"
Sep 11 17:46:43.157092 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.157051    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-jff2m" podStartSLOduration=1.923505747 podStartE2EDuration="3.157038642s" totalImagesPullingTime="1.233532895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:46:43.156609886 +0000 UTC m=+486.026652834" watchObservedRunningTime="2026-09-11 17:46:43.157038642 +0000 UTC m=+486.027081587"
Sep 11 17:46:43.404329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.404247    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:43.504748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:43.504713    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:54.132754 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.132724    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"
Sep 11 17:46:54.133139 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.132787    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:54.192146 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.192119    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:54.192323 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.192276    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-jff2m" podUID="02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" containerName="limitador" containerID="cri-o://c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e" gracePeriod=30
Sep 11 17:46:54.738794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.738772    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:54.852697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.852615    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-config-file\") pod \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") "
Sep 11 17:46:54.852697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.852671    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-546ht\" (UniqueName: \"kubernetes.io/projected/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-kube-api-access-546ht\") pod \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\" (UID: \"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e\") "
Sep 11 17:46:54.852961 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.852937    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-config-file" (OuterVolumeSpecName: "config-file") pod "02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" (UID: "02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:46:54.854697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.854678    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-kube-api-access-546ht" (OuterVolumeSpecName: "kube-api-access-546ht") pod "02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" (UID: "02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e"). InnerVolumeSpecName "kube-api-access-546ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:46:54.953886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.953853    2582 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-config-file\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:46:54.953886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:54.953883    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-546ht\" (UniqueName: \"kubernetes.io/projected/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e-kube-api-access-546ht\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:46:55.180623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.180526    2582 generic.go:356] "Generic (PLEG): container finished" podID="02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" containerID="c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e" exitCode=0
Sep 11 17:46:55.180623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.180580    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jff2m"
Sep 11 17:46:55.180623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.180605    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jff2m" event={"ID":"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e","Type":"ContainerDied","Data":"c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e"}
Sep 11 17:46:55.181178 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.180647    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jff2m" event={"ID":"02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e","Type":"ContainerDied","Data":"5d6edd75b816a61cacec167de148d0e74d3586fb814524e79c0e35a21c4584f4"}
Sep 11 17:46:55.181178 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.180667    2582 scope.go:117] "RemoveContainer" containerID="c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e"
Sep 11 17:46:55.189617 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.189601    2582 scope.go:117] "RemoveContainer" containerID="c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e"
Sep 11 17:46:55.189879 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:46:55.189859    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e\": container with ID starting with c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e not found: ID does not exist" containerID="c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e"
Sep 11 17:46:55.189928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.189888    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e"} err="failed to get container status \"c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e\": rpc error: code = NotFound desc = could not find container \"c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e\": container with ID starting with c1b7a110dfe398417f785c7466f0f7892fa86e889c082cfd3a909aaca12e028e not found: ID does not exist"
Sep 11 17:46:55.201844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.201822    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:55.206468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.206449    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jff2m"]
Sep 11 17:46:55.746742 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:55.746706    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" path="/var/lib/kubelet/pods/02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e/volumes"
Sep 11 17:46:56.692271 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:56.692238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:56.787962 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:56.787926    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:57.418176 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.418146    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"]
Sep 11 17:46:57.419364 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.419317    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" containerName="limitador"
Sep 11 17:46:57.419364 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.419362    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" containerName="limitador"
Sep 11 17:46:57.419575 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.419561    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="02dc0fe9-3f60-4f3b-a5a6-5811bf543f3e" containerName="limitador"
Sep 11 17:46:57.425008 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.424985    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.427869 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.427849    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 17:46:57.427994 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.427963    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-lb8qm\""
Sep 11 17:46:57.441392 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.441370    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 17:46:57.441554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.441500    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 17:46:57.443799 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.443775    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"]
Sep 11 17:46:57.472236 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.472212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv7sf\" (UniqueName: \"kubernetes.io/projected/98bbe7f0-81b9-4934-ad6f-7b09595fc1b7-kube-api-access-pv7sf\") pod \"postgres-69fb4594cf-cv6vq\" (UID: \"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.472361 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.472247    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/98bbe7f0-81b9-4934-ad6f-7b09595fc1b7-data\") pod \"postgres-69fb4594cf-cv6vq\" (UID: \"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.484637 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.484597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:46:57.572728 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.572706    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv7sf\" (UniqueName: \"kubernetes.io/projected/98bbe7f0-81b9-4934-ad6f-7b09595fc1b7-kube-api-access-pv7sf\") pod \"postgres-69fb4594cf-cv6vq\" (UID: \"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.572848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.572741    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/98bbe7f0-81b9-4934-ad6f-7b09595fc1b7-data\") pod \"postgres-69fb4594cf-cv6vq\" (UID: \"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.573119 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.573104    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/98bbe7f0-81b9-4934-ad6f-7b09595fc1b7-data\") pod \"postgres-69fb4594cf-cv6vq\" (UID: \"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.597990 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.597968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv7sf\" (UniqueName: \"kubernetes.io/projected/98bbe7f0-81b9-4934-ad6f-7b09595fc1b7-kube-api-access-pv7sf\") pod \"postgres-69fb4594cf-cv6vq\" (UID: \"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:57.735285 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:57.735259    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:46:58.009021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:58.008732    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"]
Sep 11 17:46:58.010859 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:46:58.010828    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bbe7f0_81b9_4934_ad6f_7b09595fc1b7.slice/crio-410705a3044893057566231990c0926dc4a53a9a942e5cb07002ccc6ce927a03 WatchSource:0}: Error finding container 410705a3044893057566231990c0926dc4a53a9a942e5cb07002ccc6ce927a03: Status 404 returned error can't find the container with id 410705a3044893057566231990c0926dc4a53a9a942e5cb07002ccc6ce927a03
Sep 11 17:46:58.196169 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:58.196136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq" event={"ID":"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7","Type":"ContainerStarted","Data":"410705a3044893057566231990c0926dc4a53a9a942e5cb07002ccc6ce927a03"}
Sep 11 17:46:58.997235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:46:58.997203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:01.212631 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:01.212598    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq" event={"ID":"98bbe7f0-81b9-4934-ad6f-7b09595fc1b7","Type":"ContainerStarted","Data":"a59b42803c91782d03baeae3ec34cabff586f2e59328958c8a97d9911e63ef37"}
Sep 11 17:47:01.212993 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:01.212663    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:47:07.245698 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:07.245668    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq"
Sep 11 17:47:07.260438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:07.260397    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-cv6vq" podStartSLOduration=7.801829425 podStartE2EDuration="10.260383019s" totalImagesPullingTime="2.458553594s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:46:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:47:01.240819836 +0000 UTC m=+504.110862774" watchObservedRunningTime="2026-09-11 17:47:07.260383019 +0000 UTC m=+510.130425964"
Sep 11 17:47:07.605309 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:07.605226    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:07.794301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:07.794267    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:14.880275 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:14.880242    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:15.571859 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:15.571825    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:15.806444 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:15.806408    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:15.967673 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:15.967581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:16.103629 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:16.103593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:19.440375 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:19.440313    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:19.614399 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:19.614338    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:41.973173 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:41.973138    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:51.893293 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:51.893255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:51.994327 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:51.994293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:53.401914 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:53.401884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:53.497788 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:53.497751    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:53.651374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:53.651319    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:53.941287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:53.941230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:54.042486 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.042451    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:54.072240 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.072210    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"]
Sep 11 17:47:54.077010 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.076991    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.078538 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.078519    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"]
Sep 11 17:47:54.079269 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.079249    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 17:47:54.079374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.079246    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-smgrn\""
Sep 11 17:47:54.082416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.082397    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.093423 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.093389    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"]
Sep 11 17:47:54.095238 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.095216    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"]
Sep 11 17:47:54.146499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.146468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:47:54.202596 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.202522    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:47:54.221944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.221915    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-v8vwn\" (UID: \"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.221944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.221943    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd77024-ebaf-4b62-8ff2-ace294c3147e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-76smh\" (UID: \"ecd77024-ebaf-4b62-8ff2-ace294c3147e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.222094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.221969    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blmjm\" (UniqueName: \"kubernetes.io/projected/ecd77024-ebaf-4b62-8ff2-ace294c3147e-kube-api-access-blmjm\") pod \"payload-pre-processing-59fbc47bf8-76smh\" (UID: \"ecd77024-ebaf-4b62-8ff2-ace294c3147e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.222094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.222049    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrqhf\" (UniqueName: \"kubernetes.io/projected/ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844-kube-api-access-lrqhf\") pod \"payload-processing-5b54b7fbb7-v8vwn\" (UID: \"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.322410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.322379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blmjm\" (UniqueName: \"kubernetes.io/projected/ecd77024-ebaf-4b62-8ff2-ace294c3147e-kube-api-access-blmjm\") pod \"payload-pre-processing-59fbc47bf8-76smh\" (UID: \"ecd77024-ebaf-4b62-8ff2-ace294c3147e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.322555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.322423    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrqhf\" (UniqueName: \"kubernetes.io/projected/ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844-kube-api-access-lrqhf\") pod \"payload-processing-5b54b7fbb7-v8vwn\" (UID: \"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.322555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.322502    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-v8vwn\" (UID: \"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.322555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.322528    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd77024-ebaf-4b62-8ff2-ace294c3147e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-76smh\" (UID: \"ecd77024-ebaf-4b62-8ff2-ace294c3147e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.323273 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.323247    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-v8vwn\" (UID: \"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.323417 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.323340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd77024-ebaf-4b62-8ff2-ace294c3147e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-76smh\" (UID: \"ecd77024-ebaf-4b62-8ff2-ace294c3147e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.334239 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.334217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blmjm\" (UniqueName: \"kubernetes.io/projected/ecd77024-ebaf-4b62-8ff2-ace294c3147e-kube-api-access-blmjm\") pod \"payload-pre-processing-59fbc47bf8-76smh\" (UID: \"ecd77024-ebaf-4b62-8ff2-ace294c3147e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.334327 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.334306    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrqhf\" (UniqueName: \"kubernetes.io/projected/ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844-kube-api-access-lrqhf\") pod \"payload-processing-5b54b7fbb7-v8vwn\" (UID: \"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.390306 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.390276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:54.397405 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.397390    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:54.669700 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.669670    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"]
Sep 11 17:47:54.672245 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:47:54.672217    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd77024_ebaf_4b62_8ff2_ace294c3147e.slice/crio-931ca4560490761c857cb0224a0f084b9b815e517019a4cf66792d85616f65f9 WatchSource:0}: Error finding container 931ca4560490761c857cb0224a0f084b9b815e517019a4cf66792d85616f65f9: Status 404 returned error can't find the container with id 931ca4560490761c857cb0224a0f084b9b815e517019a4cf66792d85616f65f9
Sep 11 17:47:54.672379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:54.672278    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"]
Sep 11 17:47:54.675879 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:47:54.675862    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba4e5b3c_c70f_41fa_bc34_bb5ba7d74844.slice/crio-a6a7e85493137572beb82ab394845fe146b067571ea5286c2d3387acb78e9dbd WatchSource:0}: Error finding container a6a7e85493137572beb82ab394845fe146b067571ea5286c2d3387acb78e9dbd: Status 404 returned error can't find the container with id a6a7e85493137572beb82ab394845fe146b067571ea5286c2d3387acb78e9dbd
Sep 11 17:47:55.478821 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:55.478725    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn" event={"ID":"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844","Type":"ContainerStarted","Data":"a6a7e85493137572beb82ab394845fe146b067571ea5286c2d3387acb78e9dbd"}
Sep 11 17:47:55.480440 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:55.480409    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh" event={"ID":"ecd77024-ebaf-4b62-8ff2-ace294c3147e","Type":"ContainerStarted","Data":"931ca4560490761c857cb0224a0f084b9b815e517019a4cf66792d85616f65f9"}
Sep 11 17:47:57.494164 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:57.494130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn" event={"ID":"ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844","Type":"ContainerStarted","Data":"ab3f791578262eedf3e1e8da06018ec80531cc93cf9432cf53fdc61dcdaf2d36"}
Sep 11 17:47:57.494690 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:57.494344    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:47:57.495717 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:57.495694    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh" event={"ID":"ecd77024-ebaf-4b62-8ff2-ace294c3147e","Type":"ContainerStarted","Data":"0a236eb0a775179d5d8ce1cad6a56743ca460fecdc1ef46963a35a603727554a"}
Sep 11 17:47:57.495895 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:57.495818    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:47:57.534328 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:57.534272    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn" podStartSLOduration=1.019074027 podStartE2EDuration="3.534253467s" totalImagesPullingTime="2.51517944s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:47:57.53171638 +0000 UTC m=+560.401759325" watchObservedRunningTime="2026-09-11 17:47:57.534253467 +0000 UTC m=+560.404296417"
Sep 11 17:47:57.569464 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:47:57.569419    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh" podStartSLOduration=1.058020779 podStartE2EDuration="3.569405276s" totalImagesPullingTime="2.511384497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:47:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:47:57.569153106 +0000 UTC m=+560.439196052" watchObservedRunningTime="2026-09-11 17:47:57.569405276 +0000 UTC m=+560.439448221"
Sep 11 17:48:00.257329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:00.257285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:04.665389 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:04.665342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:04.779205 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:04.779175    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:08.503033 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:08.502978    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-v8vwn"
Sep 11 17:48:08.503485 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:08.503298    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-76smh"
Sep 11 17:48:10.594048 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:10.594016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:11.155559 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:11.155528    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:48:16.426825 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:16.426791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:17.105401 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:17.105363    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:48:20.679028 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.678994    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"]
Sep 11 17:48:20.686324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.686301    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.688664 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.688641    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 17:48:20.688957 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.688926    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 17:48:20.688957 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.688941    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 17:48:20.689092 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.688944    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xdfcz\""
Sep 11 17:48:20.694677 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.694654    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"]
Sep 11 17:48:20.722515 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.722488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:20.728340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.728315    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68dff64a-cf98-47ea-a3da-d71725078cf3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.728455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.728378    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.728455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.728428    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.728537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.728512    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.728574 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.728546    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.728574 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.728566    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgghw\" (UniqueName: \"kubernetes.io/projected/68dff64a-cf98-47ea-a3da-d71725078cf3-kube-api-access-mgghw\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829050 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829202 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829069    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829202 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829090    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829202 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgghw\" (UniqueName: \"kubernetes.io/projected/68dff64a-cf98-47ea-a3da-d71725078cf3-kube-api-access-mgghw\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829202 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829164    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68dff64a-cf98-47ea-a3da-d71725078cf3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829444 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829216    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829542 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829515    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829564    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.829620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.829596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.831328 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.831304    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68dff64a-cf98-47ea-a3da-d71725078cf3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.831628 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.831611    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68dff64a-cf98-47ea-a3da-d71725078cf3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.837145 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.837124    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgghw\" (UniqueName: \"kubernetes.io/projected/68dff64a-cf98-47ea-a3da-d71725078cf3-kube-api-access-mgghw\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk\" (UID: \"68dff64a-cf98-47ea-a3da-d71725078cf3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:20.998480 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:20.998453    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:21.289495 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:21.289472    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"]
Sep 11 17:48:21.291878 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:48:21.291848    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68dff64a_cf98_47ea_a3da_d71725078cf3.slice/crio-47f0eb732f9020f75c6b7019a30b483d7eb6515d932293bc6494732773e51e7d WatchSource:0}: Error finding container 47f0eb732f9020f75c6b7019a30b483d7eb6515d932293bc6494732773e51e7d: Status 404 returned error can't find the container with id 47f0eb732f9020f75c6b7019a30b483d7eb6515d932293bc6494732773e51e7d
Sep 11 17:48:21.627587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:21.627502    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk" event={"ID":"68dff64a-cf98-47ea-a3da-d71725078cf3","Type":"ContainerStarted","Data":"47f0eb732f9020f75c6b7019a30b483d7eb6515d932293bc6494732773e51e7d"}
Sep 11 17:48:21.891187 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:21.891088    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:48:22.635448 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:22.635364    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk" event={"ID":"68dff64a-cf98-47ea-a3da-d71725078cf3","Type":"ContainerStarted","Data":"6f452ec4003712e41b3cf5fa68ca434e990a9ead693e49002ba3a802282490a6"}
Sep 11 17:48:22.654942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:22.654882    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk" podStartSLOduration=1.6740683440000002 podStartE2EDuration="2.654861642s" totalImagesPullingTime="980.793298ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:48:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:48:22.652428222 +0000 UTC m=+585.522471161" watchObservedRunningTime="2026-09-11 17:48:22.654861642 +0000 UTC m=+585.524904588"
Sep 11 17:48:30.849085 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:30.849048    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:30.998709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:30.998673    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:30.998898 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:30.998761    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:31.011133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:31.011109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:31.689554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:31.689523    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk"
Sep 11 17:48:32.004688 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:32.004648    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:48:37.580444 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:37.580413    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:48:37.580968 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:37.580520    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:48:38.487201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.487169    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"]
Sep 11 17:48:38.490931 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.490908    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.493209 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.493189    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 11 17:48:38.501801 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.501779    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"]
Sep 11 17:48:38.568623 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.568584    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.568806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.568654    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31478caa-4a36-4fad-9dd7-93c88b2964d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.568806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.568698    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.568806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.568725    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.568806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.568766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.568806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.568794    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s46r7\" (UniqueName: \"kubernetes.io/projected/31478caa-4a36-4fad-9dd7-93c88b2964d5-kube-api-access-s46r7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.574814 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.574787    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:38.669559 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.669529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.669950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.669576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31478caa-4a36-4fad-9dd7-93c88b2964d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.669950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.669609    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.669950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.669638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.669950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.669657    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.669950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.669672    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s46r7\" (UniqueName: \"kubernetes.io/projected/31478caa-4a36-4fad-9dd7-93c88b2964d5-kube-api-access-s46r7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.670191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.670001    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.670191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.670024    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.670191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.670068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.671928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.671897    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/31478caa-4a36-4fad-9dd7-93c88b2964d5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.672284 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.672262    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/31478caa-4a36-4fad-9dd7-93c88b2964d5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.678234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.678213    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s46r7\" (UniqueName: \"kubernetes.io/projected/31478caa-4a36-4fad-9dd7-93c88b2964d5-kube-api-access-s46r7\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7\" (UID: \"31478caa-4a36-4fad-9dd7-93c88b2964d5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:38.804278 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:38.804202    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:39.078836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:39.078813    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"]
Sep 11 17:48:39.080689 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:48:39.080661    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31478caa_4a36_4fad_9dd7_93c88b2964d5.slice/crio-0a39a6c3174472902050fe440b9fe3e34eb2d6a0cb683f1617e3be7500a85cd0 WatchSource:0}: Error finding container 0a39a6c3174472902050fe440b9fe3e34eb2d6a0cb683f1617e3be7500a85cd0: Status 404 returned error can't find the container with id 0a39a6c3174472902050fe440b9fe3e34eb2d6a0cb683f1617e3be7500a85cd0
Sep 11 17:48:39.720718 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:39.720621    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7" event={"ID":"31478caa-4a36-4fad-9dd7-93c88b2964d5","Type":"ContainerStarted","Data":"347e94e2e6a042c036079d4a493bbf85387979559ae9c9ba8dfe711d17fb9ba6"}
Sep 11 17:48:39.720718 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:39.720698    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7" event={"ID":"31478caa-4a36-4fad-9dd7-93c88b2964d5","Type":"ContainerStarted","Data":"0a39a6c3174472902050fe440b9fe3e34eb2d6a0cb683f1617e3be7500a85cd0"}
Sep 11 17:48:39.741201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:39.741144    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7" podStartSLOduration=1.5662405590000001 podStartE2EDuration="1.741126472s" totalImagesPullingTime="174.885913ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:48:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:48:39.740369234 +0000 UTC m=+602.610412172" watchObservedRunningTime="2026-09-11 17:48:39.741126472 +0000 UTC m=+602.611169418"
Sep 11 17:48:40.096204 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:40.096170    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:48:44.332106 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:44.332069    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:48:46.127746 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:46.127704    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:48:48.804549 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:48.804519    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:48.804971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:48.804581    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:48.817014 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:48.816991    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:48:49.777719 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:48:49.777689    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7"
Sep 11 17:49:21.425304 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:49:21.425271    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:49:23.596879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:49:23.596843    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:49:37.641207 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:49:37.641154    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:49:37.743726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:49:37.743692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:49:52.497532 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:49:52.497495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:49:52.554379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:49:52.554324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:27.086114 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:27.086077    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:27.153026 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:27.152953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:42.289693 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:42.289660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:42.389756 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:42.389725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:43.550808 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:43.550769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:43.642323 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:43.642285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.235915 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.235883    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.339843 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.339806    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.434807 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.434778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.628926 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.628840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.759184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.759146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.787204 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.787172    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"]
Sep 11 17:50:59.790905 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.790878    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.796839 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.796814    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 17:50:59.796965 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.796814    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-4ab253-openshift-default-dockercfg-7q9pj\""
Sep 11 17:50:59.831372 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.831334    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"]
Sep 11 17:50:59.866484 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.866453    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:50:59.960723 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960648    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-data\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.960723 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-socket\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.960912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960728    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-istio-token\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.960912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-credential-socket\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.960912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb540766-20db-4486-8989-ae0e2b6de57c-istiod-ca-cert\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.960912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960841    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-envoy\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.960912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-kube-api-access-2hdh4\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.961135 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.960926    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb540766-20db-4486-8989-ae0e2b6de57c-istio-podinfo\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.961135 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.961019    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-certs\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:50:59.973634 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:50:59.973596    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:00.061892 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.061860    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb540766-20db-4486-8989-ae0e2b6de57c-istio-podinfo\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.061901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-certs\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.061930    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-data\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.061953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-socket\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.061979    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-istio-token\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062010    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-credential-socket\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062033    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb540766-20db-4486-8989-ae0e2b6de57c-istiod-ca-cert\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-envoy\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062092    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-kube-api-access-2hdh4\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062367    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-certs\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062634 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062611    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-socket\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062785 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062765    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-credential-socket\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.062930 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.062904    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-data\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.063063 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.063044    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb540766-20db-4486-8989-ae0e2b6de57c-istiod-ca-cert\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.064481 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.064447    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-envoy\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.064649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.064633    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb540766-20db-4486-8989-ae0e2b6de57c-istio-podinfo\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.074210 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.074182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:00.074901 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.074882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-istio-token\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.075273 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.075254    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-kube-api-access-2hdh4\") pod \"e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") " pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.103539 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.103515    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:00.382421 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.382397    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"]
Sep 11 17:51:00.385107 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:00.385078    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb540766_20db_4486_8989_ae0e2b6de57c.slice/crio-fa14dcf8404746c5f34c8f8277c84d9700b30820c8d87cf0ca143ae43d1fc42f WatchSource:0}: Error finding container fa14dcf8404746c5f34c8f8277c84d9700b30820c8d87cf0ca143ae43d1fc42f: Status 404 returned error can't find the container with id fa14dcf8404746c5f34c8f8277c84d9700b30820c8d87cf0ca143ae43d1fc42f
Sep 11 17:51:00.387133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.387117    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 17:51:00.402566 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.402533    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" event={"ID":"cb540766-20db-4486-8989-ae0e2b6de57c","Type":"ContainerStarted","Data":"fa14dcf8404746c5f34c8f8277c84d9700b30820c8d87cf0ca143ae43d1fc42f"}
Sep 11 17:51:00.431634 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.431592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:00.725387 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.725273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:00.828842 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:00.828805    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:01.300673 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:01.300637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:02.687528 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:02.687488    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:02.784820 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:02.784788    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:02.885752 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:02.885714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:03.290274 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:03.290239    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:03.290412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:03.290321    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:03.290412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:03.290364    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:03.313462 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:03.313435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:51:03.421340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:03.421304    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" event={"ID":"cb540766-20db-4486-8989-ae0e2b6de57c","Type":"ContainerStarted","Data":"65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"}
Sep 11 17:51:03.447164 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:03.447100    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podStartSLOduration=1.54439494 podStartE2EDuration="4.447084414s" totalImagesPullingTime="2.902689474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:50:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:03.445671551 +0000 UTC m=+746.315714498" watchObservedRunningTime="2026-09-11 17:51:03.447084414 +0000 UTC m=+746.317127359"
Sep 11 17:51:04.066402 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.066361    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:04.104333 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.104301    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:04.109391 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.109365    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:04.229678 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.229647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:04.426546 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.426471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:04.427416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.427397    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:04.941666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:04.941599    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:05.048425 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.048383    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:05.162806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.162772    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"]
Sep 11 17:51:05.166548 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.166531    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.170076 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.170053    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-286c27-dockercfg-25lbx\""
Sep 11 17:51:05.170592 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.170573    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-286c27\""
Sep 11 17:51:05.179636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.179609    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"]
Sep 11 17:51:05.239507 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.239470    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"]
Sep 11 17:51:05.243098 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.243070    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.258023 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.257996    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"]
Sep 11 17:51:05.309039 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.309005    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22wd5\" (UniqueName: \"kubernetes.io/projected/d934c21b-959a-44bc-940b-7227316f73cc-kube-api-access-22wd5\") pod \"payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.309190 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.309099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d934c21b-959a-44bc-940b-7227316f73cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.410005 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.409972    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22wd5\" (UniqueName: \"kubernetes.io/projected/d934c21b-959a-44bc-940b-7227316f73cc-kube-api-access-22wd5\") pod \"payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.410188 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.410066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d934c21b-959a-44bc-940b-7227316f73cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.410188 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.410101    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91898659-44fa-480c-a599-90fe5b0f2359-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.410188 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.410162    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbbxj\" (UniqueName: \"kubernetes.io/projected/91898659-44fa-480c-a599-90fe5b0f2359-kube-api-access-rbbxj\") pod \"payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.410856 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.410829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d934c21b-959a-44bc-940b-7227316f73cc-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.420519 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.420496    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22wd5\" (UniqueName: \"kubernetes.io/projected/d934c21b-959a-44bc-940b-7227316f73cc-kube-api-access-22wd5\") pod \"payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.476841 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.476812    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:05.497845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.497814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:05.510794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.510768    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91898659-44fa-480c-a599-90fe5b0f2359-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.510794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.510821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbbxj\" (UniqueName: \"kubernetes.io/projected/91898659-44fa-480c-a599-90fe5b0f2359-kube-api-access-rbbxj\") pod \"payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.511957 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.511934    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91898659-44fa-480c-a599-90fe5b0f2359-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.520902 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.520876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbbxj\" (UniqueName: \"kubernetes.io/projected/91898659-44fa-480c-a599-90fe5b0f2359-kube-api-access-rbbxj\") pod \"payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.554100 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.553904    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:05.601677 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.601633    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:05.704776 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.704744    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:05.762652 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.762629    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"]
Sep 11 17:51:05.764882 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:05.764846    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd934c21b_959a_44bc_940b_7227316f73cc.slice/crio-7a0499aafac8df47543db514077fcbde4d8979dae3a321dfdc7c9b1a6093c871 WatchSource:0}: Error finding container 7a0499aafac8df47543db514077fcbde4d8979dae3a321dfdc7c9b1a6093c871: Status 404 returned error can't find the container with id 7a0499aafac8df47543db514077fcbde4d8979dae3a321dfdc7c9b1a6093c871
Sep 11 17:51:05.833505 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:05.833477    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"]
Sep 11 17:51:05.835490 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:05.835462    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91898659_44fa_480c_a599_90fe5b0f2359.slice/crio-885a04a1a56776a60c95df01dc7096ff4742d09f1e8a7dfbdaf933c661c67329 WatchSource:0}: Error finding container 885a04a1a56776a60c95df01dc7096ff4742d09f1e8a7dfbdaf933c661c67329: Status 404 returned error can't find the container with id 885a04a1a56776a60c95df01dc7096ff4742d09f1e8a7dfbdaf933c661c67329
Sep 11 17:51:06.158867 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.158710    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:06.257131 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.256915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:06.333688 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.333658    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"]
Sep 11 17:51:06.337157 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.337138    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.344531 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.344468    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-715253\""
Sep 11 17:51:06.358851 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.358823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"]
Sep 11 17:51:06.362602 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.362478    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:06.416161 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.416082    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"]
Sep 11 17:51:06.419685 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.419666    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.422313 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.422289    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-715253-dockercfg-2mt5q\""
Sep 11 17:51:06.433252 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.433227    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"]
Sep 11 17:51:06.442115 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.442062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm" event={"ID":"d934c21b-959a-44bc-940b-7227316f73cc","Type":"ContainerStarted","Data":"650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f"}
Sep 11 17:51:06.442115 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.442111    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm" event={"ID":"d934c21b-959a-44bc-940b-7227316f73cc","Type":"ContainerStarted","Data":"7a0499aafac8df47543db514077fcbde4d8979dae3a321dfdc7c9b1a6093c871"}
Sep 11 17:51:06.442744 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.442729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:06.445396 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.445212    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb" event={"ID":"91898659-44fa-480c-a599-90fe5b0f2359","Type":"ContainerStarted","Data":"c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42"}
Sep 11 17:51:06.445396 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.445291    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:06.445396 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.445314    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb" event={"ID":"91898659-44fa-480c-a599-90fe5b0f2359","Type":"ContainerStarted","Data":"885a04a1a56776a60c95df01dc7096ff4742d09f1e8a7dfbdaf933c661c67329"}
Sep 11 17:51:06.476785 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.476741    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb" podStartSLOduration=1.476725839 podStartE2EDuration="1.476725839s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:06.47326776 +0000 UTC m=+749.343310705" watchObservedRunningTime="2026-09-11 17:51:06.476725839 +0000 UTC m=+749.346768783"
Sep 11 17:51:06.490355 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.490312    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm" podStartSLOduration=1.490300076 podStartE2EDuration="1.490300076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:06.489010739 +0000 UTC m=+749.359053685" watchObservedRunningTime="2026-09-11 17:51:06.490300076 +0000 UTC m=+749.360343020"
Sep 11 17:51:06.520925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.520899    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhdj9\" (UniqueName: \"kubernetes.io/projected/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-kube-api-access-bhdj9\") pod \"payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.521035 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.520932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d827cffe-2253-499c-ad89-6334adeb302b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.521121 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.521098    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.521187 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.521171    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pnfk\" (UniqueName: \"kubernetes.io/projected/d827cffe-2253-499c-ad89-6334adeb302b-kube-api-access-7pnfk\") pod \"payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.622067 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.622020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.622215 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.622111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pnfk\" (UniqueName: \"kubernetes.io/projected/d827cffe-2253-499c-ad89-6334adeb302b-kube-api-access-7pnfk\") pod \"payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.622291 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.622253    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhdj9\" (UniqueName: \"kubernetes.io/projected/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-kube-api-access-bhdj9\") pod \"payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.622291 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.622276    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d827cffe-2253-499c-ad89-6334adeb302b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.622956 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.622933    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.623037 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.622968    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d827cffe-2253-499c-ad89-6334adeb302b-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.631400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.631376    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pnfk\" (UniqueName: \"kubernetes.io/projected/d827cffe-2253-499c-ad89-6334adeb302b-kube-api-access-7pnfk\") pod \"payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") " pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.635950 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.635927    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhdj9\" (UniqueName: \"kubernetes.io/projected/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-kube-api-access-bhdj9\") pod \"payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.650844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.650818    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:06.722901 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.722865    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:06.745384 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.745260    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:06.932952 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:06.932925    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"]
Sep 11 17:51:06.933906 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:06.933881    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd827cffe_2253_499c_ad89_6334adeb302b.slice/crio-27baf523371d4c079d223d19c1063650a9ab9b36659f95ee81bc6f3f26607a4a WatchSource:0}: Error finding container 27baf523371d4c079d223d19c1063650a9ab9b36659f95ee81bc6f3f26607a4a: Status 404 returned error can't find the container with id 27baf523371d4c079d223d19c1063650a9ab9b36659f95ee81bc6f3f26607a4a
Sep 11 17:51:07.027488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.027462    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"]
Sep 11 17:51:07.029780 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:07.029752    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b883d7d_6e20_4682_a7ee_dde9a2bb9267.slice/crio-5ab5a012a49a5e1332810d7bbc1fd692be7c441e6561ba250b4da908c7fcaf6a WatchSource:0}: Error finding container 5ab5a012a49a5e1332810d7bbc1fd692be7c441e6561ba250b4da908c7fcaf6a: Status 404 returned error can't find the container with id 5ab5a012a49a5e1332810d7bbc1fd692be7c441e6561ba250b4da908c7fcaf6a
Sep 11 17:51:07.453487 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.453449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn" event={"ID":"d827cffe-2253-499c-ad89-6334adeb302b","Type":"ContainerStarted","Data":"b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e"}
Sep 11 17:51:07.453935 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.453512    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn" event={"ID":"d827cffe-2253-499c-ad89-6334adeb302b","Type":"ContainerStarted","Data":"27baf523371d4c079d223d19c1063650a9ab9b36659f95ee81bc6f3f26607a4a"}
Sep 11 17:51:07.454132 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.454064    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:07.455794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.455767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp" event={"ID":"3b883d7d-6e20-4682-a7ee-dde9a2bb9267","Type":"ContainerStarted","Data":"e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61"}
Sep 11 17:51:07.455879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.455832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp" event={"ID":"3b883d7d-6e20-4682-a7ee-dde9a2bb9267","Type":"ContainerStarted","Data":"5ab5a012a49a5e1332810d7bbc1fd692be7c441e6561ba250b4da908c7fcaf6a"}
Sep 11 17:51:07.455942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.455862    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:07.474318 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.474269    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn" podStartSLOduration=1.4742524590000001 podStartE2EDuration="1.474252459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:07.472553647 +0000 UTC m=+750.342596594" watchObservedRunningTime="2026-09-11 17:51:07.474252459 +0000 UTC m=+750.344295404"
Sep 11 17:51:07.490879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.490841    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp" podStartSLOduration=1.490827246 podStartE2EDuration="1.490827246s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:07.489124155 +0000 UTC m=+750.359167101" watchObservedRunningTime="2026-09-11 17:51:07.490827246 +0000 UTC m=+750.360870192"
Sep 11 17:51:07.900371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:07.900311    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:08.184088 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:08.183993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:08.548043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:08.548006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:08.642127 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:08.642094    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:08.742507 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:08.742469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.056751 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.056716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.170507 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.170470    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.313786 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.313707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.640772 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.640677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.736199 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.736169    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.839669 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.839639    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:09.970036 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:09.969957    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:10.235379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.235330    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:10.341429 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.341389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:10.401092 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.401053    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"]
Sep 11 17:51:10.405019 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.404996    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.407622 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.407597    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w4-4ab253\""
Sep 11 17:51:10.422120 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.422099    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"]
Sep 11 17:51:10.440458 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.440398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:10.561259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.561169    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c0b81a9-78e6-48d2-b55a-119556f9a879-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.561259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.561233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6t86\" (UniqueName: \"kubernetes.io/projected/3c0b81a9-78e6-48d2-b55a-119556f9a879-kube-api-access-f6t86\") pod \"payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.662304 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.662263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c0b81a9-78e6-48d2-b55a-119556f9a879-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.662735 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.662335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6t86\" (UniqueName: \"kubernetes.io/projected/3c0b81a9-78e6-48d2-b55a-119556f9a879-kube-api-access-f6t86\") pod \"payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.663192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.663168    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c0b81a9-78e6-48d2-b55a-119556f9a879-plugins-config-volume\") pod \"payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.674657 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.674634    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6t86\" (UniqueName: \"kubernetes.io/projected/3c0b81a9-78e6-48d2-b55a-119556f9a879-kube-api-access-f6t86\") pod \"payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") " pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.715476 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.715449    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:10.992275 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:10.992243    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"]
Sep 11 17:51:10.993546 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:10.993516    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0b81a9_78e6_48d2_b55a_119556f9a879.slice/crio-ca4bf34d0f9363a4e1c19a9f789e49d5b9edb2002d6315e604fe58f87108127f WatchSource:0}: Error finding container ca4bf34d0f9363a4e1c19a9f789e49d5b9edb2002d6315e604fe58f87108127f: Status 404 returned error can't find the container with id ca4bf34d0f9363a4e1c19a9f789e49d5b9edb2002d6315e604fe58f87108127f
Sep 11 17:51:11.479419 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:11.479388    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp" event={"ID":"3c0b81a9-78e6-48d2-b55a-119556f9a879","Type":"ContainerStarted","Data":"73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344"}
Sep 11 17:51:11.479580 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:11.479463    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp" event={"ID":"3c0b81a9-78e6-48d2-b55a-119556f9a879","Type":"ContainerStarted","Data":"ca4bf34d0f9363a4e1c19a9f789e49d5b9edb2002d6315e604fe58f87108127f"}
Sep 11 17:51:11.479580 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:11.479503    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:11.499301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:11.499250    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp" podStartSLOduration=1.49923693 podStartE2EDuration="1.49923693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:11.498267417 +0000 UTC m=+754.368310362" watchObservedRunningTime="2026-09-11 17:51:11.49923693 +0000 UTC m=+754.369279875"
Sep 11 17:51:13.811799 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:13.811762    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:14.335057 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:14.335027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:14.426675 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:14.426640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:15.077701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:15.077666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:15.194718 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:15.194679    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:15.293785 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:15.293700    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:16.534487 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:16.534453    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:17.457846 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:17.457817    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 17:51:17.501782 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:17.501728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:17.601083 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:17.601044    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:18.008759 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.008722    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:18.158515 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.158480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:18.291552 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.291472    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:18.463136 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.463101    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:51:18.463303 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.463291    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 17:51:18.701374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.701264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:18.789191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.789161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:18.822935 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:18.822906    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:51:19.005753 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:19.005722    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:19.468333 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:19.468244    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:51:19.469476 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:19.469451    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:19.568343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:19.568312    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.006800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.006765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.118120 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.118080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.226953 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.226914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.392859 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.392789    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.491879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.491847    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.563664 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.563626    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.670798 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.670711    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.804848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.804814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:20.925179 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.925104    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"]
Sep 11 17:51:20.933288 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.933267    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:20.938218 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.938192    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-50ce3a65\""
Sep 11 17:51:20.949324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.949302    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"]
Sep 11 17:51:20.955768 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.955743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf5dd\" (UniqueName: \"kubernetes.io/projected/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-kube-api-access-xf5dd\") pod \"payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:20.955925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.955904    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:20.965041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:20.964996    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:21.057032 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.056990    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:21.057429 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.057048    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf5dd\" (UniqueName: \"kubernetes.io/projected/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-kube-api-access-xf5dd\") pod \"payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:21.057722 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.057703    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:21.072412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.072373    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf5dd\" (UniqueName: \"kubernetes.io/projected/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-kube-api-access-xf5dd\") pod \"payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:21.243884 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.243855    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:21.413527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.413490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:21.515236 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.515214    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"]
Sep 11 17:51:21.517421 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:21.517391    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cee5463_f5fd_4fea_82d8_3c3e8df1e5a5.slice/crio-51383dc5fee30c8225d84af38e309329a544f85d2461c6231576cf67f349df81 WatchSource:0}: Error finding container 51383dc5fee30c8225d84af38e309329a544f85d2461c6231576cf67f349df81: Status 404 returned error can't find the container with id 51383dc5fee30c8225d84af38e309329a544f85d2461c6231576cf67f349df81
Sep 11 17:51:21.533767 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:21.533743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx" event={"ID":"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5","Type":"ContainerStarted","Data":"51383dc5fee30c8225d84af38e309329a544f85d2461c6231576cf67f349df81"}
Sep 11 17:51:22.261973 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.261932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:22.486363 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.486319    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 17:51:22.578404 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.578293    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx" event={"ID":"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5","Type":"ContainerStarted","Data":"f7273477294ab28441df1781a9308183be80bf8d1ee56936a5bd42069600cf22"}
Sep 11 17:51:22.578575 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.578494    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:22.593383 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.593303    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx" podStartSLOduration=2.593284024 podStartE2EDuration="2.593284024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:22.593086199 +0000 UTC m=+765.463129144" watchObservedRunningTime="2026-09-11 17:51:22.593284024 +0000 UTC m=+765.463326986"
Sep 11 17:51:22.619848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.619822    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:22.982877 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:22.982844    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:23.209653 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.209609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:23.333793 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.333718    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"]
Sep 11 17:51:23.348369 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.345605    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.352366 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.350657    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9aab53-openshift-default-dockercfg-jcl6q\""
Sep 11 17:51:23.358366 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.354769    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"]
Sep 11 17:51:23.377976 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.377212    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:23.382878 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.382855    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-envoy\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383017 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.382889    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-certs\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383017 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.382911    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-data\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383123 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.383065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-socket\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383123 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.383104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-istio-token\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.383137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vn55\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-kube-api-access-9vn55\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.383159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906f1033-bc85-4040-b996-c05975bc35db-istio-podinfo\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.383187    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906f1033-bc85-4040-b996-c05975bc35db-istiod-ca-cert\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.383230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.383221    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-credential-socket\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.479870 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.479820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:23.483944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.483925    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-socket\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.483953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-istio-token\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.483973    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vn55\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-kube-api-access-9vn55\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906f1033-bc85-4040-b996-c05975bc35db-istio-podinfo\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484169 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906f1033-bc85-4040-b996-c05975bc35db-istiod-ca-cert\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484245 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484223    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-credential-socket\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-envoy\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484279    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-certs\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484389    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-socket\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484615 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-data\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484686    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-certs\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484758    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-credential-socket\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.484943 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.484876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-data\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.485090 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.485001    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906f1033-bc85-4040-b996-c05975bc35db-istiod-ca-cert\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.486529 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.486510    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906f1033-bc85-4040-b996-c05975bc35db-istio-podinfo\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.486739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.486722    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-envoy\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.494284 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.494260    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-istio-token\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.494546 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.494517    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vn55\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-kube-api-access-9vn55\") pod \"e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") " pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.580479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.580438    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:23.678371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.678248    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:23.975839 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:23.975806    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906f1033_bc85_4040_b996_c05975bc35db.slice/crio-f4e81b8052a5a895e6f368657360835d38eb9c8715f4e20602c530ec1c334fc5 WatchSource:0}: Error finding container f4e81b8052a5a895e6f368657360835d38eb9c8715f4e20602c530ec1c334fc5: Status 404 returned error can't find the container with id f4e81b8052a5a895e6f368657360835d38eb9c8715f4e20602c530ec1c334fc5
Sep 11 17:51:23.976701 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.976665    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"]
Sep 11 17:51:23.978333 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.978298    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:23.978420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.978396    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:23.978460 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:23.978438    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:24.510963 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:24.510930    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:24.594572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:24.594536    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" event={"ID":"906f1033-bc85-4040-b996-c05975bc35db","Type":"ContainerStarted","Data":"9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"}
Sep 11 17:51:24.594731 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:24.594598    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" event={"ID":"906f1033-bc85-4040-b996-c05975bc35db","Type":"ContainerStarted","Data":"f4e81b8052a5a895e6f368657360835d38eb9c8715f4e20602c530ec1c334fc5"}
Sep 11 17:51:24.621315 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:24.621262    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podStartSLOduration=1.6212455000000001 podStartE2EDuration="1.6212455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:24.619022357 +0000 UTC m=+767.489065303" watchObservedRunningTime="2026-09-11 17:51:24.6212455 +0000 UTC m=+767.491288488"
Sep 11 17:51:24.678452 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:24.678419    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:25.365833 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:25.365784    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:25.630139 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:25.630049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:25.684472 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:25.684442    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:26.606369 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:26.606315    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:26.607110 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:26.607091    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:26.662021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:26.661979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:26.757372 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:26.756234    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:27.395398 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:27.395332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:27.490367 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:27.490324    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:27.626251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:27.626204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:28.027669 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:28.027608    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:28.984694 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:28.984657    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:29.062857 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:29.062826    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:29.505743 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:29.505699    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:29.606211 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:29.606175    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:29.967770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:29.967677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:30.628317 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.628226    2582 generic.go:356] "Generic (PLEG): container finished" podID="cb540766-20db-4486-8989-ae0e2b6de57c" containerID="65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022" exitCode=0
Sep 11 17:51:30.628317 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.628297    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" event={"ID":"cb540766-20db-4486-8989-ae0e2b6de57c","Type":"ContainerDied","Data":"65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"}
Sep 11 17:51:30.628836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.628713    2582 scope.go:117] "RemoveContainer" containerID="65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"
Sep 11 17:51:30.629820 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.629795    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:30.629886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.629861    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:30.629926 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.629894    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:30.889516 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:30.889421    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:31.646227 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:31.646176    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" event={"ID":"cb540766-20db-4486-8989-ae0e2b6de57c","Type":"ContainerStarted","Data":"c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395"}
Sep 11 17:51:31.763480 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:31.763425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:32.104480 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:32.104441    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:32.105730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:32.105701    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 11 17:51:32.105874 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:32.105761    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 11 17:51:32.573424 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:32.573387    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:33.104849 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:33.104817    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 11 17:51:33.105280 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:33.104885    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 11 17:51:33.386912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:33.386822    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:33.585942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:33.585913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:33.988303 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:33.988248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:34.104289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.104258    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 11 17:51:34.104479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.104323    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 11 17:51:34.124066 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.124035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:34.249538 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.249453    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:34.399858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.399820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:34.426643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.426616    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:34.502381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.502286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:34.782376 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:34.782269    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:35.104579 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:35.104489    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 11 17:51:35.104579 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:35.104542    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 11 17:51:35.194453 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:35.194422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:35.983652 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:35.983618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:36.104128 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:36.104087    2582 patch_prober.go:28] interesting pod/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 11 17:51:36.104325 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:36.104157    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 11 17:51:36.692581 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:36.692551    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:37.108140 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:37.108107    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:37.109021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:37.109001    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 17:51:37.493671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:37.493640    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:38.146343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:38.146307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:39.053679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:39.053647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:39.775237 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:39.775210    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:39.943806 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:39.943771    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"]
Sep 11 17:51:39.947688 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:39.947670    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:39.950176 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:39.950157    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:51:39.966486 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:39.960457    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"]
Sep 11 17:51:40.048746 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.048670    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95859f26-cc30-4589-b2cb-93564502e90e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp\" (UID: \"95859f26-cc30-4589-b2cb-93564502e90e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:40.149710 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.149676    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95859f26-cc30-4589-b2cb-93564502e90e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp\" (UID: \"95859f26-cc30-4589-b2cb-93564502e90e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:40.150336 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.150314    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95859f26-cc30-4589-b2cb-93564502e90e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp\" (UID: \"95859f26-cc30-4589-b2cb-93564502e90e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:40.258445 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.258402    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:40.507946 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.507913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:51:40.527180 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.527155    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"]
Sep 11 17:51:40.528772 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:40.528745    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95859f26_cc30_4589_b2cb_93564502e90e.slice/crio-8a097de650bfa0cde2b281809ef31b6c66312ef3aafc21f9adf89abc6e35636b WatchSource:0}: Error finding container 8a097de650bfa0cde2b281809ef31b6c66312ef3aafc21f9adf89abc6e35636b: Status 404 returned error can't find the container with id 8a097de650bfa0cde2b281809ef31b6c66312ef3aafc21f9adf89abc6e35636b
Sep 11 17:51:40.705320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.705236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp" event={"ID":"95859f26-cc30-4589-b2cb-93564502e90e","Type":"ContainerStarted","Data":"8a097de650bfa0cde2b281809ef31b6c66312ef3aafc21f9adf89abc6e35636b"}
Sep 11 17:51:40.783093 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:40.783052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:41.372924 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:41.372888    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:41.477222 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:41.477183    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:41.621821 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:41.621783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:42.362664 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:42.362581    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:43.187696 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:43.187609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:43.995587 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:43.995555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:44.729086 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:44.729052    2582 generic.go:356] "Generic (PLEG): container finished" podID="95859f26-cc30-4589-b2cb-93564502e90e" containerID="ac801f28df95a4a6bdcae7452a65451cf7630140392d5f86f867485feef4851a" exitCode=0
Sep 11 17:51:44.729235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:44.729140    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp" event={"ID":"95859f26-cc30-4589-b2cb-93564502e90e","Type":"ContainerDied","Data":"ac801f28df95a4a6bdcae7452a65451cf7630140392d5f86f867485feef4851a"}
Sep 11 17:51:44.760934 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:44.760837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:45.576815 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:45.576780    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:45.907532 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:45.907507    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:46.005565 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:46.005533    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95859f26-cc30-4589-b2cb-93564502e90e-maas-api-service-ca\") pod \"95859f26-cc30-4589-b2cb-93564502e90e\" (UID: \"95859f26-cc30-4589-b2cb-93564502e90e\") "
Sep 11 17:51:46.005890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:46.005866    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95859f26-cc30-4589-b2cb-93564502e90e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "95859f26-cc30-4589-b2cb-93564502e90e" (UID: "95859f26-cc30-4589-b2cb-93564502e90e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:51:46.107034 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:46.106999    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/95859f26-cc30-4589-b2cb-93564502e90e-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:51:46.742634 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:46.742600    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"
Sep 11 17:51:46.743131 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:46.742625    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp" event={"ID":"95859f26-cc30-4589-b2cb-93564502e90e","Type":"ContainerDied","Data":"8a097de650bfa0cde2b281809ef31b6c66312ef3aafc21f9adf89abc6e35636b"}
Sep 11 17:51:46.743131 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:46.742685    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a097de650bfa0cde2b281809ef31b6c66312ef3aafc21f9adf89abc6e35636b"
Sep 11 17:51:49.946589 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:49.946555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:50.765955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:50.765921    2582 generic.go:356] "Generic (PLEG): container finished" podID="906f1033-bc85-4040-b996-c05975bc35db" containerID="9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc" exitCode=0
Sep 11 17:51:50.766110 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:50.765988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" event={"ID":"906f1033-bc85-4040-b996-c05975bc35db","Type":"ContainerDied","Data":"9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"}
Sep 11 17:51:50.766426 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:50.766412    2582 scope.go:117] "RemoveContainer" containerID="9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"
Sep 11 17:51:50.767609 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:50.767580    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:50.767675 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:50.767647    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:50.767712 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:50.767682    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:51.773847 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:51.773808    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" event={"ID":"906f1033-bc85-4040-b996-c05975bc35db","Type":"ContainerStarted","Data":"e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167"}
Sep 11 17:51:52.678956 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:52.678915    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:52.680317 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:52.680283    2582 patch_prober.go:28] interesting pod/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 17:51:52.680474 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:52.680378    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 17:51:53.678766 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:53.678731    2582 patch_prober.go:28] interesting pod/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 17:51:53.679171 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:53.678785    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 17:51:54.678757 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:54.678716    2582 patch_prober.go:28] interesting pod/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 17:51:54.678968 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:54.678785    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 17:51:55.679488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:55.679457    2582 patch_prober.go:28] interesting pod/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 17:51:55.679956 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:55.679512    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 17:51:55.886867 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:55.886828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:56.502936 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.502882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:56.538234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.538201    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"]
Sep 11 17:51:56.538763 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.538732    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx" podUID="2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" containerName="payload-pre-processing" containerID="cri-o://f7273477294ab28441df1781a9308183be80bf8d1ee56936a5bd42069600cf22" gracePeriod=30
Sep 11 17:51:56.605285 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.605250    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:56.607018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.606995    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:56.679379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.679334    2582 patch_prober.go:28] interesting pod/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Sep 11 17:51:56.679539 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.679417    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Sep 11 17:51:56.704193 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.704156    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:56.728517 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.728481    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"]
Sep 11 17:51:56.733379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.733343    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-50ce3a65-4xcdp"]
Sep 11 17:51:56.800975 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.800944    2582 generic.go:356] "Generic (PLEG): container finished" podID="2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" containerID="f7273477294ab28441df1781a9308183be80bf8d1ee56936a5bd42069600cf22" exitCode=0
Sep 11 17:51:56.801095 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.801008    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx" event={"ID":"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5","Type":"ContainerDied","Data":"f7273477294ab28441df1781a9308183be80bf8d1ee56936a5bd42069600cf22"}
Sep 11 17:51:56.801095 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.801060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx" event={"ID":"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5","Type":"ContainerDied","Data":"51383dc5fee30c8225d84af38e309329a544f85d2461c6231576cf67f349df81"}
Sep 11 17:51:56.801095 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.801084    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51383dc5fee30c8225d84af38e309329a544f85d2461c6231576cf67f349df81"
Sep 11 17:51:56.803080 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.803055    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:56.815761 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.815737    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:56.906690 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.906662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:56.907613 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.907593    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-plugins-config-volume\") pod \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") "
Sep 11 17:51:56.907697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.907682    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xf5dd\" (UniqueName: \"kubernetes.io/projected/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-kube-api-access-xf5dd\") pod \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\" (UID: \"2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5\") "
Sep 11 17:51:56.907980 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.907958    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" (UID: "2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:51:56.909728 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:56.909708    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-kube-api-access-xf5dd" (OuterVolumeSpecName: "kube-api-access-xf5dd") pod "2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" (UID: "2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5"). InnerVolumeSpecName "kube-api-access-xf5dd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:51:57.008433 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.008384    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xf5dd\" (UniqueName: \"kubernetes.io/projected/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-kube-api-access-xf5dd\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:51:57.008433 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.008431    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:51:57.159149 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.159076    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:57.439046 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.438954    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:57.542517 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.542490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:57.638556 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.638523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:57.682912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.682884    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:57.683853 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.683834    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:51:57.750197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.750158    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95859f26-cc30-4589-b2cb-93564502e90e" path="/var/lib/kubelet/pods/95859f26-cc30-4589-b2cb-93564502e90e/volumes"
Sep 11 17:51:57.806192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.806169    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"
Sep 11 17:51:57.825509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.825480    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"]
Sep 11 17:51:57.834378 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.834342    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-50ce3a65-77bf66d46-dpbtx"]
Sep 11 17:51:57.933588 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.933557    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"]
Sep 11 17:51:57.934021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.934003    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95859f26-cc30-4589-b2cb-93564502e90e" containerName="revoke-keys"
Sep 11 17:51:57.934021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.934022    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="95859f26-cc30-4589-b2cb-93564502e90e" containerName="revoke-keys"
Sep 11 17:51:57.934184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.934052    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" containerName="payload-pre-processing"
Sep 11 17:51:57.934184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.934058    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" containerName="payload-pre-processing"
Sep 11 17:51:57.934184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.934125    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" containerName="payload-pre-processing"
Sep 11 17:51:57.934184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.934135    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="95859f26-cc30-4589-b2cb-93564502e90e" containerName="revoke-keys"
Sep 11 17:51:57.940112 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.940074    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:57.943133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.943110    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-3f5d8d77-gw-openshift-default-dockercfg-bvd67\""
Sep 11 17:51:57.954558 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.954491    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"]
Sep 11 17:51:57.984886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:57.984853    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:58.017124 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017050    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-certs\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017124 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-envoy\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017124 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f259l\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-kube-api-access-f259l\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017167    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-socket\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017298    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istiod-ca-cert\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-credential-socket\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-token\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017635 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017438    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-podinfo\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.017635 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.017533    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-data\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118529 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-certs\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118711 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118544    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-envoy\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118711 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118575    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f259l\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-kube-api-access-f259l\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118711 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118603    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-socket\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118711 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118677    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istiod-ca-cert\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118719    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-credential-socket\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118745    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-token\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-podinfo\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.118928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-data\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.119145 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.118940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-certs\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.119328 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.119302    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-credential-socket\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.119427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.119326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-socket\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.119576 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.119553    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istiod-ca-cert\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.119624 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.119574    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-data\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.120959 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.120939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-envoy\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.121259 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.121242    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-podinfo\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.135226 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.135198    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-token\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.135588 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.135566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f259l\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-kube-api-access-f259l\") pod \"e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") " pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.218405 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.218366    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:58.258002 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.257971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:58.407363 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.407326    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:58.554643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.554610    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"]
Sep 11 17:51:58.556202 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:51:58.556167    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e71252_a56d_4ffd_90fa_a3133772e7ea.slice/crio-629853c117476b7d5da6e1d69db296d06b025af250ab6b0f67674ced237a4f67 WatchSource:0}: Error finding container 629853c117476b7d5da6e1d69db296d06b025af250ab6b0f67674ced237a4f67: Status 404 returned error can't find the container with id 629853c117476b7d5da6e1d69db296d06b025af250ab6b0f67674ced237a4f67
Sep 11 17:51:58.558679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.558642    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:58.558770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.558707    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:58.558770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.558736    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:51:58.603838 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.603808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:58.814161 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.814074    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" event={"ID":"f6e71252-a56d-4ffd-90fa-a3133772e7ea","Type":"ContainerStarted","Data":"4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"}
Sep 11 17:51:58.814161 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.814136    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" event={"ID":"f6e71252-a56d-4ffd-90fa-a3133772e7ea","Type":"ContainerStarted","Data":"629853c117476b7d5da6e1d69db296d06b025af250ab6b0f67674ced237a4f67"}
Sep 11 17:51:58.841597 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:58.841538    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podStartSLOduration=1.841516068 podStartE2EDuration="1.841516068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:51:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:51:58.84068011 +0000 UTC m=+801.710723064" watchObservedRunningTime="2026-09-11 17:51:58.841516068 +0000 UTC m=+801.711559014"
Sep 11 17:51:59.034359 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.034326    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:51:59.258894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.258858    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:59.264613 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.264589    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:59.747462 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.747426    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5" path="/var/lib/kubelet/pods/2cee5463-f5fd-4fea-82d8-3c3e8df1e5a5/volumes"
Sep 11 17:51:59.818975 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.818947    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:59.819933 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.819902    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:51:59.892905 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:51:59.892873    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:00.474217 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:00.474181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:00.576318 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:00.576290    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:00.843040 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:00.843009    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:01.650090 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:01.650055    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:02.506372 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:02.506303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:02.525943 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:02.525913    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:52:02.593618 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:02.593586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:02.941608 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:02.941520    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:03.185763 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:03.185729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:03.314887 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:03.314857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:03.798867 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:03.798828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:04.878888 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:04.878857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:05.528182 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:05.528134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:06.184165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:06.184132    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:19.990604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:19.990562    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:20.597244 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:20.597206    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:52:21.952499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:21.952397    2582 generic.go:356] "Generic (PLEG): container finished" podID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerID="4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443" exitCode=0
Sep 11 17:52:21.952499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:21.952468    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" event={"ID":"f6e71252-a56d-4ffd-90fa-a3133772e7ea","Type":"ContainerDied","Data":"4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"}
Sep 11 17:52:21.953018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:21.952868    2582 scope.go:117] "RemoveContainer" containerID="4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"
Sep 11 17:52:21.954157 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:21.954129    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:52:21.954239 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:21.954198    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:52:21.954239 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:21.954230    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:52:22.965851 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:22.965811    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" event={"ID":"f6e71252-a56d-4ffd-90fa-a3133772e7ea","Type":"ContainerStarted","Data":"5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041"}
Sep 11 17:52:23.258671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:23.258626    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:52:23.258860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:23.258767    2582 patch_prober.go:28] interesting pod/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:52:23.258860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:23.258811    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:52:24.258927 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:24.258890    2582 patch_prober.go:28] interesting pod/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:52:24.259537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:24.259503    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:52:24.857183 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:24.857149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:25.259551 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.259096    2582 patch_prober.go:28] interesting pod/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:52:25.259551 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.259158    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:52:25.335670 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.335632    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"]
Sep 11 17:52:25.339488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.339459    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:25.341881 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.341854    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:52:25.351050 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.350525    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"]
Sep 11 17:52:25.392800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.392741    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb7e6ad-3f2c-49f5-bc24-178914ab587f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc\" (UID: \"edb7e6ad-3f2c-49f5-bc24-178914ab587f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:25.494094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.494060    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb7e6ad-3f2c-49f5-bc24-178914ab587f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc\" (UID: \"edb7e6ad-3f2c-49f5-bc24-178914ab587f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:25.494734 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.494706    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb7e6ad-3f2c-49f5-bc24-178914ab587f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc\" (UID: \"edb7e6ad-3f2c-49f5-bc24-178914ab587f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:25.656665 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.656575    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:25.769620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.769551    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:25.932559 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.932537    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"]
Sep 11 17:52:25.934584 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:52:25.934559    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedb7e6ad_3f2c_49f5_bc24_178914ab587f.slice/crio-a90c84aa53b48e6fba2a901b5b844209fec56a3015876013ed59b79f27f5b497 WatchSource:0}: Error finding container a90c84aa53b48e6fba2a901b5b844209fec56a3015876013ed59b79f27f5b497: Status 404 returned error can't find the container with id a90c84aa53b48e6fba2a901b5b844209fec56a3015876013ed59b79f27f5b497
Sep 11 17:52:25.983996 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:25.983970    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc" event={"ID":"edb7e6ad-3f2c-49f5-bc24-178914ab587f","Type":"ContainerStarted","Data":"a90c84aa53b48e6fba2a901b5b844209fec56a3015876013ed59b79f27f5b497"}
Sep 11 17:52:26.259255 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.259211    2582 patch_prober.go:28] interesting pod/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:52:26.259556 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.259277    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:52:26.442511 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.442477    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:26.756421 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.756382    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"]
Sep 11 17:52:26.778489 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.778458    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:26.781233 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.781204    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"]
Sep 11 17:52:26.781405 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.781312    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.783612 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.783588    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"openshift-service-ca.crt\""
Sep 11 17:52:26.783767 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.783588    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"kube-root-ca.crt\""
Sep 11 17:52:26.784513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.784272    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"simulator-w4-4ab253-kserve-self-signed-certs\""
Sep 11 17:52:26.784513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.784274    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"default-dockercfg-8n74d\""
Sep 11 17:52:26.806572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.806550    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-dshm\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.806667 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.806634    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-model-cache\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.806667 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.806659    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-home\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.806759 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.806749    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tls-certs\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.806796 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.806766    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tmp-dir\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.806830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.806802    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8b77t\" (UniqueName: \"kubernetes.io/projected/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-kube-api-access-8b77t\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.907938 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.907898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tls-certs\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.907938 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.907937    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tmp-dir\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.907982    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8b77t\" (UniqueName: \"kubernetes.io/projected/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-kube-api-access-8b77t\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.908015    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-dshm\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.908077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-model-cache\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.908111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-home\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908526 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.908504    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tmp-dir\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.908548    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-model-cache\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.908648 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.908594    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-home\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.910502 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.910473    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-dshm\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.910603 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.910580    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tls-certs\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.915382 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.915361    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8b77t\" (UniqueName: \"kubernetes.io/projected/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-kube-api-access-8b77t\") pod \"simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") " pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:26.995953 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.995917    2582 generic.go:356] "Generic (PLEG): container finished" podID="edb7e6ad-3f2c-49f5-bc24-178914ab587f" containerID="f6f0a72f8616057e0eeb249958d3b16a57810640dcc3a868bfd858d797274844" exitCode=0
Sep 11 17:52:26.996103 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:26.996007    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc" event={"ID":"edb7e6ad-3f2c-49f5-bc24-178914ab587f","Type":"ContainerDied","Data":"f6f0a72f8616057e0eeb249958d3b16a57810640dcc3a868bfd858d797274844"}
Sep 11 17:52:27.093593 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:27.093502    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:27.261548 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:27.261511    2582 patch_prober.go:28] interesting pod/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Sep 11 17:52:27.261849 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:27.261820    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Sep 11 17:52:27.303881 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:27.303841    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:27.391830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:27.391801    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"]
Sep 11 17:52:27.393298 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:52:27.393274    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod292e1415_a3af_4e48_98a7_1b2d1d1e76b5.slice/crio-b6c3adf2102d1e41d9c19e292a9d850f41bdfb9a5761c1e21e36edebd40f18f1 WatchSource:0}: Error finding container b6c3adf2102d1e41d9c19e292a9d850f41bdfb9a5761c1e21e36edebd40f18f1: Status 404 returned error can't find the container with id b6c3adf2102d1e41d9c19e292a9d850f41bdfb9a5761c1e21e36edebd40f18f1
Sep 11 17:52:28.003359 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.003305    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs" event={"ID":"292e1415-a3af-4e48-98a7-1b2d1d1e76b5","Type":"ContainerStarted","Data":"0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd"}
Sep 11 17:52:28.003826 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.003372    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs" event={"ID":"292e1415-a3af-4e48-98a7-1b2d1d1e76b5","Type":"ContainerStarted","Data":"b6c3adf2102d1e41d9c19e292a9d850f41bdfb9a5761c1e21e36edebd40f18f1"}
Sep 11 17:52:28.020087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.019959    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs" podStartSLOduration=2.019935165 podStartE2EDuration="2.019935165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:52:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:52:28.019278572 +0000 UTC m=+830.889321516" watchObservedRunningTime="2026-09-11 17:52:28.019935165 +0000 UTC m=+830.889978129"
Sep 11 17:52:28.080850 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.080740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:28.169071 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.169040    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:28.222056 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.222025    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb7e6ad-3f2c-49f5-bc24-178914ab587f-maas-api-service-ca\") pod \"edb7e6ad-3f2c-49f5-bc24-178914ab587f\" (UID: \"edb7e6ad-3f2c-49f5-bc24-178914ab587f\") "
Sep 11 17:52:28.222376 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.222335    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edb7e6ad-3f2c-49f5-bc24-178914ab587f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "edb7e6ad-3f2c-49f5-bc24-178914ab587f" (UID: "edb7e6ad-3f2c-49f5-bc24-178914ab587f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:28.263230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.263164    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:52:28.263494 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.263475    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:52:28.264114 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.264099    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:52:28.323891 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.323671    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/edb7e6ad-3f2c-49f5-bc24-178914ab587f-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:28.879611 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:28.879574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:29.010731 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:29.010702    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"
Sep 11 17:52:29.011129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:29.010743    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc" event={"ID":"edb7e6ad-3f2c-49f5-bc24-178914ab587f","Type":"ContainerDied","Data":"a90c84aa53b48e6fba2a901b5b844209fec56a3015876013ed59b79f27f5b497"}
Sep 11 17:52:29.011129 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:29.010826    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a90c84aa53b48e6fba2a901b5b844209fec56a3015876013ed59b79f27f5b497"
Sep 11 17:52:29.563407 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:29.563374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:30.260691 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:30.260655    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.171091 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:31.171053    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.758766 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:31.758733    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.862871 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:31.862830    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.958146 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:31.958110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:31.975253 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:31.975224    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"]
Sep 11 17:52:31.978375 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:31.978336    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-3f5d8d77-rrsxc"]
Sep 11 17:52:32.023515 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:32.023432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:32.137601 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:32.137567    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"]
Sep 11 17:52:32.137836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:32.137811    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy" containerID="cri-o://5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041" gracePeriod=30
Sep 11 17:52:32.146507 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:32.146477    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:32.475600 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:32.475501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:32.628738 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:32.628702    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:33.004199 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:33.004164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:33.286720 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:33.286643    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:33.445435 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:33.445402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:33.748823 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:33.748789    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="edb7e6ad-3f2c-49f5-bc24-178914ab587f" path="/var/lib/kubelet/pods/edb7e6ad-3f2c-49f5-bc24-178914ab587f/volumes"
Sep 11 17:52:33.967420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:33.967385    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:34.003014 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:34.002943    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:52:35.120758 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:35.120716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:35.677340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:35.677296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:35.769046 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:35.769003    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:36.176413 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:36.176374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:36.909595 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:36.909560    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:37.093825 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.093799    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:37.093996 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.093840    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:37.106824 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.106801    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:37.404322 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.404294    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:52:37.514837 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514813    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istiod-ca-cert\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514846    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-credential-socket\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f259l\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-kube-api-access-f259l\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514894    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-podinfo\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514922    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-certs\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514964    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-envoy\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.514979    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-socket\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.514999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515000    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-token\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.515390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515043    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-data\") pod \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\" (UID: \"f6e71252-a56d-4ffd-90fa-a3133772e7ea\") "
Sep 11 17:52:37.515390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515111    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:37.515390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515292    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:37.515390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515331    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:37.515598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515423    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.515598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515442    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.515598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515536    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:37.515598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.515553    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:52:37.517201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.517176    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:37.517323 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.517231    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:52:37.517561 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.517529    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:52:37.517655 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.517591    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-kube-api-access-f259l" (OuterVolumeSpecName: "kube-api-access-f259l") pod "f6e71252-a56d-4ffd-90fa-a3133772e7ea" (UID: "f6e71252-a56d-4ffd-90fa-a3133772e7ea"). InnerVolumeSpecName "kube-api-access-f259l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:52:37.616849 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616821    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.616849 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616844    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.616849 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616853    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f259l\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-kube-api-access-f259l\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.617053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616862    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.617053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616871    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.617053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616881    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.617053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.616888    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6e71252-a56d-4ffd-90fa-a3133772e7ea-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:52:37.747198 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.747162    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:37.850820 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:37.850732    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:38.066733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.066694    2582 generic.go:356] "Generic (PLEG): container finished" podID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerID="5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041" exitCode=0
Sep 11 17:52:38.066903 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.066733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" event={"ID":"f6e71252-a56d-4ffd-90fa-a3133772e7ea","Type":"ContainerDied","Data":"5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041"}
Sep 11 17:52:38.066903 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.066760    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"
Sep 11 17:52:38.066903 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.066832    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq" event={"ID":"f6e71252-a56d-4ffd-90fa-a3133772e7ea","Type":"ContainerDied","Data":"629853c117476b7d5da6e1d69db296d06b025af250ab6b0f67674ced237a4f67"}
Sep 11 17:52:38.066903 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.066863    2582 scope.go:117] "RemoveContainer" containerID="5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041"
Sep 11 17:52:38.076299 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.076285    2582 scope.go:117] "RemoveContainer" containerID="4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"
Sep 11 17:52:38.081757 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.081729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 17:52:38.083938 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.083917    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"]
Sep 11 17:52:38.086835 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.086818    2582 scope.go:117] "RemoveContainer" containerID="5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041"
Sep 11 17:52:38.087110 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:52:38.087090    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041\": container with ID starting with 5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041 not found: ID does not exist" containerID="5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041"
Sep 11 17:52:38.087190 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.087121    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041"} err="failed to get container status \"5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041\": rpc error: code = NotFound desc = could not find container \"5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041\": container with ID starting with 5dbc9cba00443af3ad07ed8df63477b4fd627bd40fe89d4a07886b8faca7e041 not found: ID does not exist"
Sep 11 17:52:38.087190 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.087147    2582 scope.go:117] "RemoveContainer" containerID="4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"
Sep 11 17:52:38.087485 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:52:38.087466    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443\": container with ID starting with 4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443 not found: ID does not exist" containerID="4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"
Sep 11 17:52:38.087530 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.087492    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443"} err="failed to get container status \"4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443\": rpc error: code = NotFound desc = could not find container \"4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443\": container with ID starting with 4244190f53a78591c0b408409853f6bf242627ff280a85c9227ba9545ad54443 not found: ID does not exist"
Sep 11 17:52:38.092977 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.092956    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-3f5d8d77-gw-openshift-default-68876b8cdd-27htq"]
Sep 11 17:52:38.229367 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.229320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:38.319756 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.319721    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:38.658416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:38.658319    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:39.473909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:39.473877    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:39.746978 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:39.746942    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" path="/var/lib/kubelet/pods/f6e71252-a56d-4ffd-90fa-a3133772e7ea/volumes"
Sep 11 17:52:40.271472 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:40.271441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:40.941272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:40.941237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:41.676897 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:41.676860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:52:53.295895 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:53.295853    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:52:59.762858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:52:59.762820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:00.591393 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:00.591340    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:01.500334 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.500301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:01.528107 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528079    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"]
Sep 11 17:53:01.528573 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528557    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edb7e6ad-3f2c-49f5-bc24-178914ab587f" containerName="revoke-keys"
Sep 11 17:53:01.528647 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528576    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="edb7e6ad-3f2c-49f5-bc24-178914ab587f" containerName="revoke-keys"
Sep 11 17:53:01.528647 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528602    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy"
Sep 11 17:53:01.528647 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528611    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy"
Sep 11 17:53:01.528743 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528714    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy"
Sep 11 17:53:01.528743 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528721    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="edb7e6ad-3f2c-49f5-bc24-178914ab587f" containerName="revoke-keys"
Sep 11 17:53:01.528743 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.528730    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy"
Sep 11 17:53:01.533327 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.533309    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:01.535618 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.535597    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:53:01.540006 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.539979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2263f85-8142-4186-a2e2-a4a75c199ce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj\" (UID: \"b2263f85-8142-4186-a2e2-a4a75c199ce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:01.542019 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.541997    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"]
Sep 11 17:53:01.641165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.641133    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2263f85-8142-4186-a2e2-a4a75c199ce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj\" (UID: \"b2263f85-8142-4186-a2e2-a4a75c199ce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:01.641765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.641745    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2263f85-8142-4186-a2e2-a4a75c199ce4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj\" (UID: \"b2263f85-8142-4186-a2e2-a4a75c199ce4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:01.844913 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:01.844826    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:02.117132 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:02.117107    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"]
Sep 11 17:53:02.119015 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:53:02.118990    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2263f85_8142_4186_a2e2_a4a75c199ce4.slice/crio-6a923a7de5f523b839d786454d94aeb19b28294fbe803e2377a5e12a53badca7 WatchSource:0}: Error finding container 6a923a7de5f523b839d786454d94aeb19b28294fbe803e2377a5e12a53badca7: Status 404 returned error can't find the container with id 6a923a7de5f523b839d786454d94aeb19b28294fbe803e2377a5e12a53badca7
Sep 11 17:53:02.204184 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:02.204153    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj" event={"ID":"b2263f85-8142-4186-a2e2-a4a75c199ce4","Type":"ContainerStarted","Data":"6a923a7de5f523b839d786454d94aeb19b28294fbe803e2377a5e12a53badca7"}
Sep 11 17:53:02.377416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:02.377320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:03.212246 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:03.212042    2582 generic.go:356] "Generic (PLEG): container finished" podID="b2263f85-8142-4186-a2e2-a4a75c199ce4" containerID="b5ff858317a4143359cd990af6ba1e6787f945dc92f2a09c155c73187db366d0" exitCode=0
Sep 11 17:53:03.212246 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:03.212090    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj" event={"ID":"b2263f85-8142-4186-a2e2-a4a75c199ce4","Type":"ContainerDied","Data":"b5ff858317a4143359cd990af6ba1e6787f945dc92f2a09c155c73187db366d0"}
Sep 11 17:53:03.305967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:03.305932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:04.192021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:04.191974    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:04.360667 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:04.360645    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:04.467847 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:04.467769    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2263f85-8142-4186-a2e2-a4a75c199ce4-maas-api-service-ca\") pod \"b2263f85-8142-4186-a2e2-a4a75c199ce4\" (UID: \"b2263f85-8142-4186-a2e2-a4a75c199ce4\") "
Sep 11 17:53:04.468122 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:04.468099    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2263f85-8142-4186-a2e2-a4a75c199ce4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b2263f85-8142-4186-a2e2-a4a75c199ce4" (UID: "b2263f85-8142-4186-a2e2-a4a75c199ce4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:04.568932 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:04.568898    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b2263f85-8142-4186-a2e2-a4a75c199ce4-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:53:04.942429 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:04.942373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:05.223971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:05.223893    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"
Sep 11 17:53:05.223971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:05.223918    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj" event={"ID":"b2263f85-8142-4186-a2e2-a4a75c199ce4","Type":"ContainerDied","Data":"6a923a7de5f523b839d786454d94aeb19b28294fbe803e2377a5e12a53badca7"}
Sep 11 17:53:05.223971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:05.223959    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6a923a7de5f523b839d786454d94aeb19b28294fbe803e2377a5e12a53badca7"
Sep 11 17:53:05.778218 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:05.778181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:06.480469 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:06.480433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:07.177606 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:07.177574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:07.800526 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:07.800494    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:07.901212 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:07.901177    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:08.000098 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:08.000071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:08.048870 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:08.048835    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"]
Sep 11 17:53:08.051733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:08.051666    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-feb821e2-wxvjj"]
Sep 11 17:53:08.065606 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:08.065583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:08.478020 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:08.477933    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:08.548234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:08.548196    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:09.268088 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:09.268056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:09.412766 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:09.412724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:09.713576 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:09.713492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:09.748256 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:09.748222    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2263f85-8142-4186-a2e2-a4a75c199ce4" path="/var/lib/kubelet/pods/b2263f85-8142-4186-a2e2-a4a75c199ce4/volumes"
Sep 11 17:53:10.047969 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:10.047937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:10.323942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:10.323846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:10.521766 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:10.521729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:11.339927 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:11.339894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:12.257882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:12.257846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:12.922138 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:12.922099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:13.017398 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:13.017342    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:13.116931 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:13.116900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:13.889725 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:13.889692    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:14.757000 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:14.756965    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:15.110373 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:15.110329    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:53:15.641200 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:15.641170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:16.394021 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:16.393988    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:17.151944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:17.151911    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:17.890732 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:17.890680    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:17.990553 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:17.990518    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:18.091847 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:18.091819    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:18.383996 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:18.383947    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:18.490710 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:18.490670    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:33.795695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:33.795624    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:53:37.594866 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:37.594827    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:37.618569 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:37.618534    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:53:37.618850 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:37.618832    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:53:40.532321 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:40.532276    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:41.527492 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:41.527456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:42.383873 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.383827    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:42.947173 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.947137    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"]
Sep 11 17:53:42.947621 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.947606    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy"
Sep 11 17:53:42.947621 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.947621    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6e71252-a56d-4ffd-90fa-a3133772e7ea" containerName="istio-proxy"
Sep 11 17:53:42.947747 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.947633    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2263f85-8142-4186-a2e2-a4a75c199ce4" containerName="revoke-keys"
Sep 11 17:53:42.947747 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.947639    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2263f85-8142-4186-a2e2-a4a75c199ce4" containerName="revoke-keys"
Sep 11 17:53:42.947747 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.947714    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2263f85-8142-4186-a2e2-a4a75c199ce4" containerName="revoke-keys"
Sep 11 17:53:42.952011 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.951989    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:42.954086 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.954065    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:53:42.958382 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:42.958337    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"]
Sep 11 17:53:43.033764 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.033735    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f096d7-d4db-455a-8f62-2c97acb467f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5f786284-45th8\" (UID: \"c5f096d7-d4db-455a-8f62-2c97acb467f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:43.135111 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.135070    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f096d7-d4db-455a-8f62-2c97acb467f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5f786284-45th8\" (UID: \"c5f096d7-d4db-455a-8f62-2c97acb467f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:43.135745 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.135725    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f096d7-d4db-455a-8f62-2c97acb467f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-5f786284-45th8\" (UID: \"c5f096d7-d4db-455a-8f62-2c97acb467f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:43.265607 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.265576    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:43.402702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.402666    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:43.534848 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.534822    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"]
Sep 11 17:53:43.537069 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:53:43.537043    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5f096d7_d4db_455a_8f62_2c97acb467f1.slice/crio-0d9757ca59444f207c38d456240fd5a576fb95141907791812d7259fb499e2dd WatchSource:0}: Error finding container 0d9757ca59444f207c38d456240fd5a576fb95141907791812d7259fb499e2dd: Status 404 returned error can't find the container with id 0d9757ca59444f207c38d456240fd5a576fb95141907791812d7259fb499e2dd
Sep 11 17:53:43.929361 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.929317    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"]
Sep 11 17:53:43.932894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.932877    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:43.940457 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.940310    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"]
Sep 11 17:53:43.942967 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:43.942944    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-37899e-2pd29\" (UID: \"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:44.044119 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.044084    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-37899e-2pd29\" (UID: \"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:44.044702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.044681    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-autores-37899e-2pd29\" (UID: \"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:44.245709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.245673    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:44.290942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.290910    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:44.448784 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.448752    2582 generic.go:356] "Generic (PLEG): container finished" podID="c5f096d7-d4db-455a-8f62-2c97acb467f1" containerID="e08785876ac59ca7e4fb7bd21a7772ff3e6592455ead60cec7b139fcd9001081" exitCode=0
Sep 11 17:53:44.449165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.448850    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8" event={"ID":"c5f096d7-d4db-455a-8f62-2c97acb467f1","Type":"ContainerDied","Data":"e08785876ac59ca7e4fb7bd21a7772ff3e6592455ead60cec7b139fcd9001081"}
Sep 11 17:53:44.449165 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.448910    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8" event={"ID":"c5f096d7-d4db-455a-8f62-2c97acb467f1","Type":"ContainerStarted","Data":"0d9757ca59444f207c38d456240fd5a576fb95141907791812d7259fb499e2dd"}
Sep 11 17:53:44.517681 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:44.517658    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"]
Sep 11 17:53:44.519017 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:53:44.518984    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3f4a68c_4fe0_4cc1_84eb_13bba52773b3.slice/crio-822421083d7c69852c9974d2d400ccf64783a232a5ffd7010957afea08de4e63 WatchSource:0}: Error finding container 822421083d7c69852c9974d2d400ccf64783a232a5ffd7010957afea08de4e63: Status 404 returned error can't find the container with id 822421083d7c69852c9974d2d400ccf64783a232a5ffd7010957afea08de4e63
Sep 11 17:53:45.233294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.233261    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:45.455494 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.455403    2582 generic.go:356] "Generic (PLEG): container finished" podID="e3f4a68c-4fe0-4cc1-84eb-13bba52773b3" containerID="fd12515582791f10b6bf3d8f3e34e4c441d1e18dc688a31fea434b8e34b9288f" exitCode=0
Sep 11 17:53:45.455494 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.455474    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29" event={"ID":"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3","Type":"ContainerDied","Data":"fd12515582791f10b6bf3d8f3e34e4c441d1e18dc688a31fea434b8e34b9288f"}
Sep 11 17:53:45.455944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.455514    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29" event={"ID":"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3","Type":"ContainerStarted","Data":"822421083d7c69852c9974d2d400ccf64783a232a5ffd7010957afea08de4e63"}
Sep 11 17:53:45.583609 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.583585    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:45.658412 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.658382    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f096d7-d4db-455a-8f62-2c97acb467f1-maas-api-service-ca\") pod \"c5f096d7-d4db-455a-8f62-2c97acb467f1\" (UID: \"c5f096d7-d4db-455a-8f62-2c97acb467f1\") "
Sep 11 17:53:45.658707 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.658683    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5f096d7-d4db-455a-8f62-2c97acb467f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c5f096d7-d4db-455a-8f62-2c97acb467f1" (UID: "c5f096d7-d4db-455a-8f62-2c97acb467f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:45.759636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.759611    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5f096d7-d4db-455a-8f62-2c97acb467f1-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:53:45.917142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:45.917109    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:46.462312 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.462286    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"
Sep 11 17:53:46.462795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.462320    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8" event={"ID":"c5f096d7-d4db-455a-8f62-2c97acb467f1","Type":"ContainerDied","Data":"0d9757ca59444f207c38d456240fd5a576fb95141907791812d7259fb499e2dd"}
Sep 11 17:53:46.462795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.462381    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d9757ca59444f207c38d456240fd5a576fb95141907791812d7259fb499e2dd"
Sep 11 17:53:46.601580 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.601558    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:46.666565 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.666535    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3-maas-api-service-ca\") pod \"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3\" (UID: \"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3\") "
Sep 11 17:53:46.666864 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.666841    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e3f4a68c-4fe0-4cc1-84eb-13bba52773b3" (UID: "e3f4a68c-4fe0-4cc1-84eb-13bba52773b3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:53:46.767863 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:46.767832    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:53:47.475320 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:47.475277    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29" event={"ID":"e3f4a68c-4fe0-4cc1-84eb-13bba52773b3","Type":"ContainerDied","Data":"822421083d7c69852c9974d2d400ccf64783a232a5ffd7010957afea08de4e63"}
Sep 11 17:53:47.475735 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:47.475289    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"
Sep 11 17:53:47.475735 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:47.475398    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="822421083d7c69852c9974d2d400ccf64783a232a5ffd7010957afea08de4e63"
Sep 11 17:53:47.895925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:47.895890    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:48.594527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:48.594490    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:49.445069 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:49.445025    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:50.290095 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:50.290059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:53.024558 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:53.024514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:53.125517 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:53.125482    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.068194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.068156    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.650455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.650424    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.728925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.728887    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.802025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.801989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.896547 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.896502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:54.915123 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.915042    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"]
Sep 11 17:53:54.919478 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:54.919450    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-5f786284-45th8"]
Sep 11 17:53:55.009268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.009233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.113489 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.113440    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.142611 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.142574    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"]
Sep 11 17:53:55.146641 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.146605    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-autores-37899e-2pd29"]
Sep 11 17:53:55.204465 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.204378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.453467 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.453413    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.556744 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.556709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.714078 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.714044    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:53:55.748289 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.748257    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5f096d7-d4db-455a-8f62-2c97acb467f1" path="/var/lib/kubelet/pods/c5f096d7-d4db-455a-8f62-2c97acb467f1/volumes"
Sep 11 17:53:55.748658 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.748641    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3f4a68c-4fe0-4cc1-84eb-13bba52773b3" path="/var/lib/kubelet/pods/e3f4a68c-4fe0-4cc1-84eb-13bba52773b3/volumes"
Sep 11 17:53:55.826884 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.826789    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:55.921027 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:55.920991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:56.026684 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:56.026646    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:56.509380 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:56.509337    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:57.470570 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:57.470540    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:57.607205 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:57.607170    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:58.382939 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:58.382900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:53:59.169182 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:53:59.169147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:00.008302 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:00.008263    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:00.622461 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:00.622431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:03.887647 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:03.887618    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:04.003094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:04.003057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:04.103142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:04.103097    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:04.175814 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:04.175737    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:04.618407 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:04.618320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:07.684677 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:07.684644    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:13.795561 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:13.795529    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:54:30.587260 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:30.587221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:33.816732 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:33.816701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:36.741649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:36.741611    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:36.857561 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:36.857530    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:36.961296 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:36.961255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:38.350489 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:38.350452    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:38.642532 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:38.642437    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:39.130212 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:39.130180    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:54:39.131704 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:39.131670    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:39.467909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:39.467811    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:39.689226 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:39.689189    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:41.961742 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:41.961709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:42.153061 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:42.153027    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:42.258465 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:42.258433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:42.972312 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:42.972283    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:44.075554 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:44.075522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:44.885868 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:44.885833    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:45.852248 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:45.852199    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:45.958426 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:45.958398    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:46.402595 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:46.402560    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:46.557875 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:46.557840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:51.756490 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756456    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"]
Sep 11 17:54:51.756879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756860    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5f096d7-d4db-455a-8f62-2c97acb467f1" containerName="revoke-keys"
Sep 11 17:54:51.756879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756872    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5f096d7-d4db-455a-8f62-2c97acb467f1" containerName="revoke-keys"
Sep 11 17:54:51.756965 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756888    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3f4a68c-4fe0-4cc1-84eb-13bba52773b3" containerName="revoke-keys"
Sep 11 17:54:51.756965 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756894    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3f4a68c-4fe0-4cc1-84eb-13bba52773b3" containerName="revoke-keys"
Sep 11 17:54:51.756965 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756956    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5f096d7-d4db-455a-8f62-2c97acb467f1" containerName="revoke-keys"
Sep 11 17:54:51.756965 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.756964    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3f4a68c-4fe0-4cc1-84eb-13bba52773b3" containerName="revoke-keys"
Sep 11 17:54:51.760575 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.760553    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.762765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.762734    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-4ab253\"/\"premium-w4-4ab253-kserve-self-signed-certs\""
Sep 11 17:54:51.768960 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.768939    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"]
Sep 11 17:54:51.861251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.861225    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-model-cache\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.861251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.861256    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5chwb\" (UniqueName: \"kubernetes.io/projected/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-kube-api-access-5chwb\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.861496 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.861306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tls-certs\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.861496 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.861417    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-dshm\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.861496 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.861441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-home\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.861610 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.861509    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tmp-dir\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.961929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.961898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tls-certs\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.961949    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-dshm\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.961964    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-home\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.962000    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tmp-dir\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.962026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-model-cache\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.962045    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5chwb\" (UniqueName: \"kubernetes.io/projected/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-kube-api-access-5chwb\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962446 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.962430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tmp-dir\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962512 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.962461    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-model-cache\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.962573 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.962523    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-home\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.964212 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.964191    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-dshm\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.964521 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.964503    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tls-certs\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:51.969535 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:51.969513    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5chwb\" (UniqueName: \"kubernetes.io/projected/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-kube-api-access-5chwb\") pod \"premium-w4-4ab253-kserve-94fbfc785-zkz7z\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") " pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:52.071146 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.071070    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:54:52.287628 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.287593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:52.344029 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.344000    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"]
Sep 11 17:54:52.347649 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:54:52.347625    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6e9993e_16ed_49e5_8b06_32aec1d81aa5.slice/crio-caea1e051ce5ac61ba7e8948effbee2ef70860abf346b36caa479bd56f653122 WatchSource:0}: Error finding container caea1e051ce5ac61ba7e8948effbee2ef70860abf346b36caa479bd56f653122: Status 404 returned error can't find the container with id caea1e051ce5ac61ba7e8948effbee2ef70860abf346b36caa479bd56f653122
Sep 11 17:54:52.843018 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.842978    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z" event={"ID":"e6e9993e-16ed-49e5-8b06-32aec1d81aa5","Type":"ContainerStarted","Data":"22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687"}
Sep 11 17:54:52.843635 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.843031    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z" event={"ID":"e6e9993e-16ed-49e5-8b06-32aec1d81aa5","Type":"ContainerStarted","Data":"caea1e051ce5ac61ba7e8948effbee2ef70860abf346b36caa479bd56f653122"}
Sep 11 17:54:52.866602 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.866548    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z" podStartSLOduration=1.866529539 podStartE2EDuration="1.866529539s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:54:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:54:52.865847803 +0000 UTC m=+975.735890750" watchObservedRunningTime="2026-09-11 17:54:52.866529539 +0000 UTC m=+975.736572496"
Sep 11 17:54:52.908988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:52.908954    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:54.299692 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:54.299648    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:54:58.792772 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:58.792740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:58.905956 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:58.905924    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:54:58.998364 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:54:58.998321    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:02.072171 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.072090    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:55:02.072171 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.072154    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:55:02.084858 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.084833    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:55:02.596315 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.596285    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"]
Sep 11 17:55:02.599841 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.599825    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.602869 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.602846    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9aab53\"/\"kube-root-ca.crt\""
Sep 11 17:55:02.603477 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.603453    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9aab53\"/\"default-dockercfg-rxnmm\""
Sep 11 17:55:02.603612 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.603509    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-9aab53\"/\"openshift-service-ca.crt\""
Sep 11 17:55:02.604708 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.604693    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-9aab53\"/\"test-model-9aab53-kserve-self-signed-certs\""
Sep 11 17:55:02.628193 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.628170    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"]
Sep 11 17:55:02.658057 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.658025    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-home\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.658192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.658071    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-tmp-dir\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.658192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.658150    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e116ab66-1575-4ac6-9bdb-63f1468311ad-tls-certs\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.658271 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.658212    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-dshm\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.658271 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.658237    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-model-cache\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.658336 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.658273    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7plgd\" (UniqueName: \"kubernetes.io/projected/e116ab66-1575-4ac6-9bdb-63f1468311ad-kube-api-access-7plgd\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.664537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.664510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:02.759571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759541    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-dshm\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.759571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759574    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-model-cache\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.759800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759596    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7plgd\" (UniqueName: \"kubernetes.io/projected/e116ab66-1575-4ac6-9bdb-63f1468311ad-kube-api-access-7plgd\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.759800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-home\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.759800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759685    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-tmp-dir\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.759800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759720    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e116ab66-1575-4ac6-9bdb-63f1468311ad-tls-certs\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.759997 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.759939    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-model-cache\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.760109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.760087    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-home\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.760172 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.760123    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-tmp-dir\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.761886 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.761868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-dshm\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.762224 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.762206    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e116ab66-1575-4ac6-9bdb-63f1468311ad-tls-certs\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.768448 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.768430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7plgd\" (UniqueName: \"kubernetes.io/projected/e116ab66-1575-4ac6-9bdb-63f1468311ad-kube-api-access-7plgd\") pod \"test-model-9aab53-kserve-cf46cc79f-nfvz6\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") " pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.910611 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.910532    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:02.913974 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:02.913696    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 17:55:03.189130 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:03.189103    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"]
Sep 11 17:55:03.191274 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:55:03.191248    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode116ab66_1575_4ac6_9bdb_63f1468311ad.slice/crio-fb423df5709940761d68a9dfea1369d9be33d46f0c16c9571ee42fd732e8f04c WatchSource:0}: Error finding container fb423df5709940761d68a9dfea1369d9be33d46f0c16c9571ee42fd732e8f04c: Status 404 returned error can't find the container with id fb423df5709940761d68a9dfea1369d9be33d46f0c16c9571ee42fd732e8f04c
Sep 11 17:55:03.909097 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:03.909058    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6" event={"ID":"e116ab66-1575-4ac6-9bdb-63f1468311ad","Type":"ContainerStarted","Data":"3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec"}
Sep 11 17:55:03.909097 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:03.909104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6" event={"ID":"e116ab66-1575-4ac6-9bdb-63f1468311ad","Type":"ContainerStarted","Data":"fb423df5709940761d68a9dfea1369d9be33d46f0c16c9571ee42fd732e8f04c"}
Sep 11 17:55:03.926975 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:03.926921    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6" podStartSLOduration=1.9269042170000001 podStartE2EDuration="1.926904217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:55:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:55:03.926098861 +0000 UTC m=+986.796141806" watchObservedRunningTime="2026-09-11 17:55:03.926904217 +0000 UTC m=+986.796947161"
Sep 11 17:55:08.308406 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:08.308343    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:08.983655 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:08.983619    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:09.607480 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:09.607436    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:09.790290 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:09.790256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:10.514417 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:10.514378    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:10.613465 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:10.613432    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:55:10.615323 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:10.615292    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:11.451645 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:11.451604    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:12.297159 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:12.297127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:12.407137 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:12.407106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:12.910690 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:12.910656    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:12.910844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:12.910706    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:12.923146 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:12.923117    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:12.973231 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:12.973201    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:55:16.531196 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:16.531154    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:16.615635 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:16.615599    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:16.707551 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:16.707492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:16.803916 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:16.803829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:16.905075 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:16.905036    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:17.602671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:17.602624    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:17.701414 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:17.701373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:17.801643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:17.801609    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:19.531717 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:19.531685    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:20.969058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:20.969022    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:21.080059 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:21.079998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:25.637499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:25.637469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:25.826004 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:25.825970    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:26.354661 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:26.354629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:28.526450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:28.526413    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:55:32.246784 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:32.246749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:32.346790 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:32.346754    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:32.504379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:32.504269    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:32.945030 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:32.944932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:33.288755 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:33.288719    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:39.404094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:39.404059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:39.648014 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:39.647979    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:39.764109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:39.764079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:40.493489 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:40.493456    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:41.234628 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:41.234592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:42.179937 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:42.179907    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:42.938445 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:42.938405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:43.802986 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:43.802953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:43.870405 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:43.870371    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:43.970044 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:43.970009    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:44.359860 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:44.359833    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:44.467827 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:44.467796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:55:54.908324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:54.908290    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:55:56.373461 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:55:56.373425    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:05.871549 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:05.871510    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:06.591659 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:06.591628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:07.598078 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:07.598041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:08.354764 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:08.354725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:09.112673 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:09.112637    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:09.896325 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:09.896291    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:10.068663 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.068630    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:10.641270 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.641236    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:10.749695 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.749662    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:10.787326 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.787295    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"]
Sep 11 17:56:10.791255 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.791234    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:10.794232 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.794210    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-104872\"/\"default-dockercfg-9cmrf\""
Sep 11 17:56:10.794400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.794252    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-104872\"/\"openshift-service-ca.crt\""
Sep 11 17:56:10.794400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.794252    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-explicit-104872\"/\"explicit-model-104872-kserve-self-signed-certs\""
Sep 11 17:56:10.794522 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.794510    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-explicit-104872\"/\"kube-root-ca.crt\""
Sep 11 17:56:10.805526 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.805502    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"]
Sep 11 17:56:10.854263 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.854230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:10.909306 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.909231    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-dshm\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:10.909306 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.909261    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-home\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:10.909306 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.909283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-model-cache\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:10.909534 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.909339    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tmp-dir\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:10.909534 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.909434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfgn7\" (UniqueName: \"kubernetes.io/projected/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-kube-api-access-sfgn7\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:10.909534 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:10.909457    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tls-certs\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010221 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010181    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-model-cache\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010221 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tmp-dir\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010505 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfgn7\" (UniqueName: \"kubernetes.io/projected/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-kube-api-access-sfgn7\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010505 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010326    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tls-certs\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010505 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010406    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-dshm\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010505 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-home\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010717 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010631    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-model-cache\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010717 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010649    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tmp-dir\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.010823 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.010752    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-home\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.013204 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.012878    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-dshm\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.013377 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.013225    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tls-certs\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.018261 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.018236    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfgn7\" (UniqueName: \"kubernetes.io/projected/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-kube-api-access-sfgn7\") pod \"explicit-model-104872-kserve-58c7d7f8fc-ppsrm\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") " pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.100862 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.100829    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:11.113369 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.113316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:11.214190 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.214107    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:11.377894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:11.377870    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"]
Sep 11 17:56:11.379868 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:56:11.379839    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f066a8c_b6de_4ea5_8db9_0a2dbe6a0652.slice/crio-d72428287786ea7ca0d1096d1186ecb4d57581bfeeae3aaf071bddedb868c6a2 WatchSource:0}: Error finding container d72428287786ea7ca0d1096d1186ecb4d57581bfeeae3aaf071bddedb868c6a2: Status 404 returned error can't find the container with id d72428287786ea7ca0d1096d1186ecb4d57581bfeeae3aaf071bddedb868c6a2
Sep 11 17:56:12.321155 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:12.321115    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm" event={"ID":"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652","Type":"ContainerStarted","Data":"acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4"}
Sep 11 17:56:12.321624 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:12.321167    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm" event={"ID":"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652","Type":"ContainerStarted","Data":"d72428287786ea7ca0d1096d1186ecb4d57581bfeeae3aaf071bddedb868c6a2"}
Sep 11 17:56:12.343004 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:12.342947    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm" podStartSLOduration=2.342924753 podStartE2EDuration="2.342924753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:56:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:56:12.340783015 +0000 UTC m=+1055.210825960" watchObservedRunningTime="2026-09-11 17:56:12.342924753 +0000 UTC m=+1055.212967702"
Sep 11 17:56:17.348845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.348799    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:17.423844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.423814    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:17.730071 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.730043    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"]
Sep 11 17:56:17.733869 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.733852    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:17.736054 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.736034    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:56:17.753731 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.753706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"]
Sep 11 17:56:17.775178 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.775153    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29069d16-dd4e-496e-846b-56d6a0451e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s\" (UID: \"29069d16-dd4e-496e-846b-56d6a0451e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:17.875857 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.875830    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29069d16-dd4e-496e-846b-56d6a0451e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s\" (UID: \"29069d16-dd4e-496e-846b-56d6a0451e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:17.876454 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:17.876426    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29069d16-dd4e-496e-846b-56d6a0451e22-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s\" (UID: \"29069d16-dd4e-496e-846b-56d6a0451e22\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:18.046955 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:18.046870    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:18.317770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:18.317743    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"]
Sep 11 17:56:18.319756 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:56:18.319728    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29069d16_dd4e_496e_846b_56d6a0451e22.slice/crio-7993f97466ed4f9a769a0e1ffef42953f70992b9a78986f70bd0eb30606b5827 WatchSource:0}: Error finding container 7993f97466ed4f9a769a0e1ffef42953f70992b9a78986f70bd0eb30606b5827: Status 404 returned error can't find the container with id 7993f97466ed4f9a769a0e1ffef42953f70992b9a78986f70bd0eb30606b5827
Sep 11 17:56:18.355601 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:18.355571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s" event={"ID":"29069d16-dd4e-496e-846b-56d6a0451e22","Type":"ContainerStarted","Data":"7993f97466ed4f9a769a0e1ffef42953f70992b9a78986f70bd0eb30606b5827"}
Sep 11 17:56:18.589863 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:18.589769    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:19.363601 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:19.363518    2582 generic.go:356] "Generic (PLEG): container finished" podID="29069d16-dd4e-496e-846b-56d6a0451e22" containerID="ed1f524cc9737e5c8c7ab5855594d9ccd79e2c5752caad82c6a9efb248463c2f" exitCode=0
Sep 11 17:56:19.363601 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:19.363583    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s" event={"ID":"29069d16-dd4e-496e-846b-56d6a0451e22","Type":"ContainerDied","Data":"ed1f524cc9737e5c8c7ab5855594d9ccd79e2c5752caad82c6a9efb248463c2f"}
Sep 11 17:56:19.465709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:19.465678    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:20.142162 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:20.142116    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:20.519738 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:20.519714    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:20.601197 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:20.601164    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29069d16-dd4e-496e-846b-56d6a0451e22-maas-api-service-ca\") pod \"29069d16-dd4e-496e-846b-56d6a0451e22\" (UID: \"29069d16-dd4e-496e-846b-56d6a0451e22\") "
Sep 11 17:56:20.601545 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:20.601520    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29069d16-dd4e-496e-846b-56d6a0451e22-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29069d16-dd4e-496e-846b-56d6a0451e22" (UID: "29069d16-dd4e-496e-846b-56d6a0451e22"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:20.702971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:20.702894    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29069d16-dd4e-496e-846b-56d6a0451e22-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:20.943113 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:20.943078    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:21.101827 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.101795    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:21.101985 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.101958    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:21.114594 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.114566    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:21.376390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.376287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s" event={"ID":"29069d16-dd4e-496e-846b-56d6a0451e22","Type":"ContainerDied","Data":"7993f97466ed4f9a769a0e1ffef42953f70992b9a78986f70bd0eb30606b5827"}
Sep 11 17:56:21.376390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.376337    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7993f97466ed4f9a769a0e1ffef42953f70992b9a78986f70bd0eb30606b5827"
Sep 11 17:56:21.376390 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.376304    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"
Sep 11 17:56:21.387896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.387867    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:21.664391 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:21.664285    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:22.327797 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:22.327761    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:23.084993 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:23.084953    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:26.106002 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:26.105962    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:27.705277 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:27.705240    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:56:28.256959 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.256922    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:28.336775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.336741    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:28.441651 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.441605    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:28.534166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.534087    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"]
Sep 11 17:56:28.540864 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.540836    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-fcdaf27d-pvg5s"]
Sep 11 17:56:28.544576 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.544548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:28.947232 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:28.947146    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:29.050762 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:29.050728    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:29.750209 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:29.750174    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29069d16-dd4e-496e-846b-56d6a0451e22" path="/var/lib/kubelet/pods/29069d16-dd4e-496e-846b-56d6a0451e22/volumes"
Sep 11 17:56:29.905765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:29.905729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:30.584943 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:30.584904    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:31.101379 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.101328    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"]
Sep 11 17:56:31.153764 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.153696    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:31.198170 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.198139    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"]
Sep 11 17:56:31.198797 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.198769    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29069d16-dd4e-496e-846b-56d6a0451e22" containerName="revoke-keys"
Sep 11 17:56:31.198797 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.198789    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="29069d16-dd4e-496e-846b-56d6a0451e22" containerName="revoke-keys"
Sep 11 17:56:31.199043 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.198940    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="29069d16-dd4e-496e-846b-56d6a0451e22" containerName="revoke-keys"
Sep 11 17:56:31.202540 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.202521    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:31.204590 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.204571    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:56:31.210809 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.210785    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"]
Sep 11 17:56:31.303429 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.303396    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a40489d-7fdc-4dd5-9427-57c92bfae6dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-104872-kxtxk\" (UID: \"5a40489d-7fdc-4dd5-9427-57c92bfae6dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:31.404891 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.404811    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a40489d-7fdc-4dd5-9427-57c92bfae6dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-104872-kxtxk\" (UID: \"5a40489d-7fdc-4dd5-9427-57c92bfae6dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:31.405479 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.405462    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a40489d-7fdc-4dd5-9427-57c92bfae6dc-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-explicit-104872-kxtxk\" (UID: \"5a40489d-7fdc-4dd5-9427-57c92bfae6dc\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:31.518113 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.518081    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:31.559086 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.559055    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:31.789535 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:31.789511    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"]
Sep 11 17:56:31.792027 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:56:31.791994    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a40489d_7fdc_4dd5_9427_57c92bfae6dc.slice/crio-08d44761eeb07989ee31c681650f90f3ed9816ec1e60f4967261fb7f9c17a7c7 WatchSource:0}: Error finding container 08d44761eeb07989ee31c681650f90f3ed9816ec1e60f4967261fb7f9c17a7c7: Status 404 returned error can't find the container with id 08d44761eeb07989ee31c681650f90f3ed9816ec1e60f4967261fb7f9c17a7c7
Sep 11 17:56:32.271865 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:32.271834    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:32.450311 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:32.450275    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk" event={"ID":"5a40489d-7fdc-4dd5-9427-57c92bfae6dc","Type":"ContainerStarted","Data":"13bb4d24b8a5aefe9ec4fd37ebf60340c4ff8a492060b5a895c98b5befd4b219"}
Sep 11 17:56:32.450489 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:32.450398    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk" event={"ID":"5a40489d-7fdc-4dd5-9427-57c92bfae6dc","Type":"ContainerStarted","Data":"08d44761eeb07989ee31c681650f90f3ed9816ec1e60f4967261fb7f9c17a7c7"}
Sep 11 17:56:32.470244 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:32.470200    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk" podStartSLOduration=1.4701860629999999 podStartE2EDuration="1.470186063s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:56:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:56:32.469024364 +0000 UTC m=+1075.339067309" watchObservedRunningTime="2026-09-11 17:56:32.470186063 +0000 UTC m=+1075.340229008"
Sep 11 17:56:33.174684 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:33.174650    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:33.457795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:33.457699    2582 generic.go:356] "Generic (PLEG): container finished" podID="5a40489d-7fdc-4dd5-9427-57c92bfae6dc" containerID="13bb4d24b8a5aefe9ec4fd37ebf60340c4ff8a492060b5a895c98b5befd4b219" exitCode=0
Sep 11 17:56:33.457795 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:33.457763    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk" event={"ID":"5a40489d-7fdc-4dd5-9427-57c92bfae6dc","Type":"ContainerDied","Data":"13bb4d24b8a5aefe9ec4fd37ebf60340c4ff8a492060b5a895c98b5befd4b219"}
Sep 11 17:56:33.967077 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:33.967041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:34.539980 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:34.539949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:34.612936 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:34.612910    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:34.738225 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:34.738197    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a40489d-7fdc-4dd5-9427-57c92bfae6dc-maas-api-service-ca\") pod \"5a40489d-7fdc-4dd5-9427-57c92bfae6dc\" (UID: \"5a40489d-7fdc-4dd5-9427-57c92bfae6dc\") "
Sep 11 17:56:34.738670 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:34.738644    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a40489d-7fdc-4dd5-9427-57c92bfae6dc-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5a40489d-7fdc-4dd5-9427-57c92bfae6dc" (UID: "5a40489d-7fdc-4dd5-9427-57c92bfae6dc"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:56:34.839525 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:34.839495    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a40489d-7fdc-4dd5-9427-57c92bfae6dc-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:35.297964 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.297917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:35.422045 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.422010    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:35.469887 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.469859    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"
Sep 11 17:56:35.470041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.469897    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk" event={"ID":"5a40489d-7fdc-4dd5-9427-57c92bfae6dc","Type":"ContainerDied","Data":"08d44761eeb07989ee31c681650f90f3ed9816ec1e60f4967261fb7f9c17a7c7"}
Sep 11 17:56:35.470041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.469965    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="08d44761eeb07989ee31c681650f90f3ed9816ec1e60f4967261fb7f9c17a7c7"
Sep 11 17:56:35.523393 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.523341    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:35.700622 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:35.700527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:36.095074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:36.095043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:36.228948 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:36.228914    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:36.331781 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:36.331748    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:41.829821 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:41.829791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:41.907720 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:41.907672    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:42.006005 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.005973    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:42.033302 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.033271    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"]
Sep 11 17:56:42.036749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.036727    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-explicit-104872-kxtxk"]
Sep 11 17:56:42.111328 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.111254    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:42.391525 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.391441    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:42.501577 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.501544    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:42.595288 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.595238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:42.793381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:42.793326    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:43.288934 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:43.288879    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:43.748216 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:43.748180    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a40489d-7fdc-4dd5-9427-57c92bfae6dc" path="/var/lib/kubelet/pods/5a40489d-7fdc-4dd5-9427-57c92bfae6dc/volumes"
Sep 11 17:56:44.429511 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:44.429480    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:45.536912 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:45.536875    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:46.003065 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.003035    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:46.120424 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.120378    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm" podUID="9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" containerName="main" containerID="cri-o://acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4" gracePeriod=45
Sep 11 17:56:46.392678 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.392654    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:46.458620 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458591    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-model-cache\") pod \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") "
Sep 11 17:56:46.458775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458633    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-dshm\") pod \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") "
Sep 11 17:56:46.458775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458699    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tmp-dir\") pod \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") "
Sep 11 17:56:46.458775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458727    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sfgn7\" (UniqueName: \"kubernetes.io/projected/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-kube-api-access-sfgn7\") pod \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") "
Sep 11 17:56:46.458775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458762    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tls-certs\") pod \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") "
Sep 11 17:56:46.458944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458802    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-home\") pod \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\" (UID: \"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652\") "
Sep 11 17:56:46.458944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458902    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" (UID: "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:46.459034 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.458996    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" (UID: "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:46.459168 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.459140    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-home" (OuterVolumeSpecName: "home") pod "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" (UID: "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:46.459248 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.459171    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:46.459248 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.459191    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:46.461042 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.461009    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-dshm" (OuterVolumeSpecName: "dshm") pod "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" (UID: "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:56:46.461042 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.461022    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-kube-api-access-sfgn7" (OuterVolumeSpecName: "kube-api-access-sfgn7") pod "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" (UID: "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652"). InnerVolumeSpecName "kube-api-access-sfgn7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:56:46.461294 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.461271    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" (UID: "9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:56:46.536716 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.536684    2582 generic.go:356] "Generic (PLEG): container finished" podID="9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" containerID="acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4" exitCode=0
Sep 11 17:56:46.536882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.536728    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"
Sep 11 17:56:46.536882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.536768    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm" event={"ID":"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652","Type":"ContainerDied","Data":"acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4"}
Sep 11 17:56:46.536882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.536818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm" event={"ID":"9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652","Type":"ContainerDied","Data":"d72428287786ea7ca0d1096d1186ecb4d57581bfeeae3aaf071bddedb868c6a2"}
Sep 11 17:56:46.536882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.536841    2582 scope.go:117] "RemoveContainer" containerID="acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4"
Sep 11 17:56:46.543598 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.543506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:46.547420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.547398    2582 scope.go:117] "RemoveContainer" containerID="acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4"
Sep 11 17:56:46.547766 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:56:46.547723    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4\": container with ID starting with acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4 not found: ID does not exist" containerID="acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4"
Sep 11 17:56:46.547874 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.547761    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4"} err="failed to get container status \"acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4\": rpc error: code = NotFound desc = could not find container \"acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4\": container with ID starting with acd32c6052ee8efcb72049fcdf0d216387f89189baa13c1f5c9b6ca7a75e8be4 not found: ID does not exist"
Sep 11 17:56:46.560181 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.560157    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sfgn7\" (UniqueName: \"kubernetes.io/projected/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-kube-api-access-sfgn7\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:46.560181 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.560179    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:46.560303 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.560190    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:46.560303 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.560197    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:56:46.566223 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.566202    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"]
Sep 11 17:56:46.575166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:46.575144    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-explicit-104872/explicit-model-104872-kserve-58c7d7f8fc-ppsrm"]
Sep 11 17:56:47.748364 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:47.748327    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" path="/var/lib/kubelet/pods/9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652/volumes"
Sep 11 17:56:49.031409 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:49.031377    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:49.136750 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:49.136709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:49.291689 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:49.291603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:49.396987 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:49.396952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:49.874813 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:49.874782    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:50.296890 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:50.296860    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:50.464509 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:50.464474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:50.782555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:50.782511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:50.860471 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:50.860443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:51.040299 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:51.040215    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:51.344988 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:51.344885    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:51.432216 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:51.432184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:51.583527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:51.583497    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:51.684302 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:51.684207    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:53.365460 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:53.365420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:54.014306 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:54.014275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:54.102752 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:54.102716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:54.202754 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:54.202703    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:54.979102 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:54.979067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:55.793673 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:55.793643    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:56.571524 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:56.571492    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:57.273423 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:57.273386    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:57.373792 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:57.373759    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:57.753609 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:57.753580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:57.850423 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:57.850386    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:57.933857 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:57.933825    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:56:59.303386 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:56:59.303340    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:57:04.634666 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:04.634629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:04.734553 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:04.734513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:20.035224 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:20.035172    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:21.129318 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:21.129279    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:21.930267 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:21.930232    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:22.811107 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:22.811067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:23.572228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:23.572193    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:24.229923 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.229886    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:24.437256 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.437221    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:24.601630 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.601522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:24.924207 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924126    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"]
Sep 11 17:57:24.924834 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924817    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a40489d-7fdc-4dd5-9427-57c92bfae6dc" containerName="revoke-keys"
Sep 11 17:57:24.924894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924838    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a40489d-7fdc-4dd5-9427-57c92bfae6dc" containerName="revoke-keys"
Sep 11 17:57:24.924894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924853    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" containerName="main"
Sep 11 17:57:24.924894 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924863    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" containerName="main"
Sep 11 17:57:24.924994 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924962    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f066a8c-b6de-4ea5-8db9-0a2dbe6a0652" containerName="main"
Sep 11 17:57:24.924994 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.924971    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a40489d-7fdc-4dd5-9427-57c92bfae6dc" containerName="revoke-keys"
Sep 11 17:57:24.928376 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.928339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:24.930879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.930858    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-7f85c3-openshift-default-dockercfg-ch62v\""
Sep 11 17:57:24.942850 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.942827    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"]
Sep 11 17:57:24.963109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:24.963079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027008    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9cnt\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-kube-api-access-s9cnt\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027070    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-credential-socket\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027148    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-socket\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027174    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istiod-ca-cert\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-certs\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-envoy\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-podinfo\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-data\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.027450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.027413    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-token\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.127931 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.127896    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-certs\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.127940    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-envoy\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.127994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-podinfo\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-data\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128067    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-token\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128109 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9cnt\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-kube-api-access-s9cnt\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128337 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128143    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-credential-socket\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128337 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-socket\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128337 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128229    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istiod-ca-cert\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128531 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128339    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-certs\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128788 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128764    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-data\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128881 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128823    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-credential-socket\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128881 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128845    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-socket\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.128999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.128936    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istiod-ca-cert\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.130739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.130712    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-podinfo\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.130878 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.130825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-envoy\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.137979 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.137954    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-token\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.138174 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.138151    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9cnt\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-kube-api-access-s9cnt\") pod \"e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") " pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.197381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.197297    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:25.240049 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.240019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:25.409983 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.409948    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:57:25.523061 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.523036    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"]
Sep 11 17:57:25.524392 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:57:25.524367    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18de25a0_645d_45eb_8cfa_a4ce4f7471e4.slice/crio-2d96759ef24843fe94fb43817132044fea6d34ec766426ddc8d7a9dcd25364da WatchSource:0}: Error finding container 2d96759ef24843fe94fb43817132044fea6d34ec766426ddc8d7a9dcd25364da: Status 404 returned error can't find the container with id 2d96759ef24843fe94fb43817132044fea6d34ec766426ddc8d7a9dcd25364da
Sep 11 17:57:25.526765 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.526730    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:57:25.526871 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.526812    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:57:25.526871 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.526860    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:57:25.773166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.773062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" event={"ID":"18de25a0-645d-45eb-8cfa-a4ce4f7471e4","Type":"ContainerStarted","Data":"2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"}
Sep 11 17:57:25.773166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.773160    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" event={"ID":"18de25a0-645d-45eb-8cfa-a4ce4f7471e4","Type":"ContainerStarted","Data":"2d96759ef24843fe94fb43817132044fea6d34ec766426ddc8d7a9dcd25364da"}
Sep 11 17:57:25.796055 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:25.795999    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podStartSLOduration=1.795982881 podStartE2EDuration="1.795982881s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:25.794740594 +0000 UTC m=+1128.664783541" watchObservedRunningTime="2026-09-11 17:57:25.795982881 +0000 UTC m=+1128.666025826"
Sep 11 17:57:26.198791 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:26.198705    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:26.240310 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:26.240278    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:26.246522 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:26.246488    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:26.780035 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:26.780005    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:26.780839 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:26.780817    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:57:27.281937 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:27.281908    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:27.903777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:27.903734    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:28.123728 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:28.123693    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:28.484942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:28.484900    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:29.594799 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:29.594765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:30.488621 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:30.488586    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:31.259798 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:31.259750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:32.214343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:32.214304    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:32.963939 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:32.963906    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:33.061215 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:33.061179    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:33.320942 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:33.320910    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:33.451342 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:33.451310    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:33.570210 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:33.570171    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:37.694221 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:37.694187    2582 scope.go:117] "RemoveContainer" containerID="f7273477294ab28441df1781a9308183be80bf8d1ee56936a5bd42069600cf22"
Sep 11 17:57:40.370615 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.370576    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"]
Sep 11 17:57:40.375458 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.375438    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.377675 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.377653    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6baa6b\"/\"kube-root-ca.crt\""
Sep 11 17:57:40.378241 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.378224    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-6baa6b\"/\"openshift-service-ca.crt\""
Sep 11 17:57:40.378315 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.378277    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6baa6b\"/\"default-dockercfg-28qdt\""
Sep 11 17:57:40.378396 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.378322    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-6baa6b\"/\"noten-model-6baa6b-kserve-self-signed-certs\""
Sep 11 17:57:40.384084 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.384048    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"]
Sep 11 17:57:40.472443 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.472408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-model-cache\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.472591 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.472446    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-dshm\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.472591 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.472473    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-tmp-dir\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.472591 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.472499    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnmf6\" (UniqueName: \"kubernetes.io/projected/4295924b-0b61-4a08-8994-9629b048fd2f-kube-api-access-xnmf6\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.472591 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.472535    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4295924b-0b61-4a08-8994-9629b048fd2f-tls-certs\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.472591 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.472564    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-home\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573310 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-model-cache\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573504 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-dshm\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573504 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573376    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-tmp-dir\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573504 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573398    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnmf6\" (UniqueName: \"kubernetes.io/projected/4295924b-0b61-4a08-8994-9629b048fd2f-kube-api-access-xnmf6\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573504 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573425    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4295924b-0b61-4a08-8994-9629b048fd2f-tls-certs\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573504 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573468    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-home\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573778 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573756    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-model-cache\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573844 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573808    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-tmp-dir\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.573904 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.573877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-home\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.575583 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.575564    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-dshm\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.575836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.575819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4295924b-0b61-4a08-8994-9629b048fd2f-tls-certs\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.582133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.582111    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnmf6\" (UniqueName: \"kubernetes.io/projected/4295924b-0b61-4a08-8994-9629b048fd2f-kube-api-access-xnmf6\") pod \"noten-model-6baa6b-kserve-5766cbf59f-r7gwf\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") " pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.685304 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.685235    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:40.764319 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.764286    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:40.962410 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:40.959978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"]
Sep 11 17:57:41.869866 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:41.869825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf" event={"ID":"4295924b-0b61-4a08-8994-9629b048fd2f","Type":"ContainerStarted","Data":"66306600fb2ca5eaba1baf46c7e852ef2d73386f6969a6c6be1191bad111e34a"}
Sep 11 17:57:41.870257 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:41.869877    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf" event={"ID":"4295924b-0b61-4a08-8994-9629b048fd2f","Type":"ContainerStarted","Data":"b4b8052b95364a8ba9998cc54dd96191bf946a8c9f2da9e9b9c1687edcd76bc6"}
Sep 11 17:57:41.888582 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:41.888534    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf" podStartSLOduration=1.888519024 podStartE2EDuration="1.888519024s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:57:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:57:41.885810003 +0000 UTC m=+1144.755852948" watchObservedRunningTime="2026-09-11 17:57:41.888519024 +0000 UTC m=+1144.758561969"
Sep 11 17:57:46.267976 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.267941    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"]
Sep 11 17:57:46.289720 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.289687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:46.420794 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.420761    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"]
Sep 11 17:57:46.424707 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.424683    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:46.427054 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.427034    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:57:46.433828 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.433805    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"]
Sep 11 17:57:46.532761 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.532686    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60dd994-82c5-440c-9959-167f1d96b4c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-715253-mb6v2\" (UID: \"b60dd994-82c5-440c-9959-167f1d96b4c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:46.603498 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.603469    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:46.633971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.633946    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60dd994-82c5-440c-9959-167f1d96b4c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-715253-mb6v2\" (UID: \"b60dd994-82c5-440c-9959-167f1d96b4c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:46.634651 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.634627    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60dd994-82c5-440c-9959-167f1d96b4c9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-715253-mb6v2\" (UID: \"b60dd994-82c5-440c-9959-167f1d96b4c9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:46.737726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:46.737693    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:47.005808 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:47.005782    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"]
Sep 11 17:57:47.007327 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:57:47.007294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60dd994_82c5_440c_9959_167f1d96b4c9.slice/crio-11cac2605088fc9e445fc7ff6e66630dbf8efaa9c7719db06c06a8ad8ba0f104 WatchSource:0}: Error finding container 11cac2605088fc9e445fc7ff6e66630dbf8efaa9c7719db06c06a8ad8ba0f104: Status 404 returned error can't find the container with id 11cac2605088fc9e445fc7ff6e66630dbf8efaa9c7719db06c06a8ad8ba0f104
Sep 11 17:57:47.474421 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:47.474382    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:47.906217 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:47.906187    2582 generic.go:356] "Generic (PLEG): container finished" podID="b60dd994-82c5-440c-9959-167f1d96b4c9" containerID="270e48e6d04c0d80955dd2c2fdce534e379ddfb9ca543d24d5749a2fe8a76dfb" exitCode=0
Sep 11 17:57:47.906374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:47.906249    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2" event={"ID":"b60dd994-82c5-440c-9959-167f1d96b4c9","Type":"ContainerDied","Data":"270e48e6d04c0d80955dd2c2fdce534e379ddfb9ca543d24d5749a2fe8a76dfb"}
Sep 11 17:57:47.906374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:47.906300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2" event={"ID":"b60dd994-82c5-440c-9959-167f1d96b4c9","Type":"ContainerStarted","Data":"11cac2605088fc9e445fc7ff6e66630dbf8efaa9c7719db06c06a8ad8ba0f104"}
Sep 11 17:57:48.192166 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:48.192079    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:48.412799 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:48.412764    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:57:48.981907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:48.981866    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:49.064305 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.064281    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:49.162648 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.162570    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60dd994-82c5-440c-9959-167f1d96b4c9-maas-api-service-ca\") pod \"b60dd994-82c5-440c-9959-167f1d96b4c9\" (UID: \"b60dd994-82c5-440c-9959-167f1d96b4c9\") "
Sep 11 17:57:49.162976 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.162956    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60dd994-82c5-440c-9959-167f1d96b4c9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b60dd994-82c5-440c-9959-167f1d96b4c9" (UID: "b60dd994-82c5-440c-9959-167f1d96b4c9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:57:49.263874 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.263842    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b60dd994-82c5-440c-9959-167f1d96b4c9-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:57:49.679041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.679006    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:49.918339 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.918297    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2" event={"ID":"b60dd994-82c5-440c-9959-167f1d96b4c9","Type":"ContainerDied","Data":"11cac2605088fc9e445fc7ff6e66630dbf8efaa9c7719db06c06a8ad8ba0f104"}
Sep 11 17:57:49.918544 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.918320    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"
Sep 11 17:57:49.918544 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:49.918375    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="11cac2605088fc9e445fc7ff6e66630dbf8efaa9c7719db06c06a8ad8ba0f104"
Sep 11 17:57:50.465174 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:50.465137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:50.686107 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:50.686068    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:50.686285 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:50.686127    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:50.699000 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:50.698975    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:50.935699 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:50.935669    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:57:51.249297 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:51.249266    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:51.968257 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:51.968225    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:52.665836 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:52.665796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:52.908065 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:52.908033    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:53.791568 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:53.791529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:54.584897 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:54.584866    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:55.471928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:55.471894    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:56.262064 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:56.262028    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:56.937146 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:56.937110    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:57.000381 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.000329    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"]
Sep 11 17:57:57.000635 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.000609    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn" podUID="d827cffe-2253-499c-ad89-6334adeb302b" containerName="payload-processing" containerID="cri-o://b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e" gracePeriod=30
Sep 11 17:57:57.010283 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.009181    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"]
Sep 11 17:57:57.010283 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.009442    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp" podUID="3b883d7d-6e20-4682-a7ee-dde9a2bb9267" containerName="payload-pre-processing" containerID="cri-o://e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61" gracePeriod=30
Sep 11 17:57:57.037028 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.036998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:57.138539 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.138502    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:57.186020 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.185960    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"]
Sep 11 17:57:57.190151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.189821    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-715253-mb6v2"]
Sep 11 17:57:57.209992 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.209881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:57.299593 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.299563    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:57:57.335713 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.335687    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:57:57.460915 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.460825    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pnfk\" (UniqueName: \"kubernetes.io/projected/d827cffe-2253-499c-ad89-6334adeb302b-kube-api-access-7pnfk\") pod \"d827cffe-2253-499c-ad89-6334adeb302b\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") "
Sep 11 17:57:57.460915 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.460892    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhdj9\" (UniqueName: \"kubernetes.io/projected/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-kube-api-access-bhdj9\") pod \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") "
Sep 11 17:57:57.461126 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.460989    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d827cffe-2253-499c-ad89-6334adeb302b-plugins-config-volume\") pod \"d827cffe-2253-499c-ad89-6334adeb302b\" (UID: \"d827cffe-2253-499c-ad89-6334adeb302b\") "
Sep 11 17:57:57.461126 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.461040    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-plugins-config-volume\") pod \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\" (UID: \"3b883d7d-6e20-4682-a7ee-dde9a2bb9267\") "
Sep 11 17:57:57.461455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.461411    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d827cffe-2253-499c-ad89-6334adeb302b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d827cffe-2253-499c-ad89-6334adeb302b" (UID: "d827cffe-2253-499c-ad89-6334adeb302b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:57:57.461455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.461433    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3b883d7d-6e20-4682-a7ee-dde9a2bb9267" (UID: "3b883d7d-6e20-4682-a7ee-dde9a2bb9267"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:57:57.463497 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.463477    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d827cffe-2253-499c-ad89-6334adeb302b-kube-api-access-7pnfk" (OuterVolumeSpecName: "kube-api-access-7pnfk") pod "d827cffe-2253-499c-ad89-6334adeb302b" (UID: "d827cffe-2253-499c-ad89-6334adeb302b"). InnerVolumeSpecName "kube-api-access-7pnfk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:57:57.463565 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.463506    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-kube-api-access-bhdj9" (OuterVolumeSpecName: "kube-api-access-bhdj9") pod "3b883d7d-6e20-4682-a7ee-dde9a2bb9267" (UID: "3b883d7d-6e20-4682-a7ee-dde9a2bb9267"). InnerVolumeSpecName "kube-api-access-bhdj9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:57:57.562657 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.562621    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pnfk\" (UniqueName: \"kubernetes.io/projected/d827cffe-2253-499c-ad89-6334adeb302b-kube-api-access-7pnfk\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:57:57.562657 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.562658    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhdj9\" (UniqueName: \"kubernetes.io/projected/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-kube-api-access-bhdj9\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:57:57.562830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.562672    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d827cffe-2253-499c-ad89-6334adeb302b-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:57:57.562830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.562687    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3b883d7d-6e20-4682-a7ee-dde9a2bb9267-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:57:57.592835 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.592801    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:57.744519 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.744466    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:57.748896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.748871    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b60dd994-82c5-440c-9959-167f1d96b4c9" path="/var/lib/kubelet/pods/b60dd994-82c5-440c-9959-167f1d96b4c9/volumes"
Sep 11 17:57:57.822521 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.822478    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 17:57:57.822684 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.822551    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 17:57:57.974298 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.974260    2582 generic.go:356] "Generic (PLEG): container finished" podID="d827cffe-2253-499c-ad89-6334adeb302b" containerID="b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e" exitCode=0
Sep 11 17:57:57.974733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.974306    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"
Sep 11 17:57:57.974733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.974362    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn" event={"ID":"d827cffe-2253-499c-ad89-6334adeb302b","Type":"ContainerDied","Data":"b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e"}
Sep 11 17:57:57.974733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.974426    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn" event={"ID":"d827cffe-2253-499c-ad89-6334adeb302b","Type":"ContainerDied","Data":"27baf523371d4c079d223d19c1063650a9ab9b36659f95ee81bc6f3f26607a4a"}
Sep 11 17:57:57.974733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.974458    2582 scope.go:117] "RemoveContainer" containerID="b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e"
Sep 11 17:57:57.975976 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.975942    2582 generic.go:356] "Generic (PLEG): container finished" podID="3b883d7d-6e20-4682-a7ee-dde9a2bb9267" containerID="e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61" exitCode=0
Sep 11 17:57:57.976074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.975976    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"
Sep 11 17:57:57.976074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.975995    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp" event={"ID":"3b883d7d-6e20-4682-a7ee-dde9a2bb9267","Type":"ContainerDied","Data":"e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61"}
Sep 11 17:57:57.976074 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.976048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp" event={"ID":"3b883d7d-6e20-4682-a7ee-dde9a2bb9267","Type":"ContainerDied","Data":"5ab5a012a49a5e1332810d7bbc1fd692be7c441e6561ba250b4da908c7fcaf6a"}
Sep 11 17:57:57.984400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.984381    2582 scope.go:117] "RemoveContainer" containerID="b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e"
Sep 11 17:57:57.984664 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:57:57.984644    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e\": container with ID starting with b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e not found: ID does not exist" containerID="b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e"
Sep 11 17:57:57.984748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.984675    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e"} err="failed to get container status \"b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e\": rpc error: code = NotFound desc = could not find container \"b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e\": container with ID starting with b8ece1476ebc6c911085cfd71d0219d65336609e304bde817e1941e81fbde91e not found: ID does not exist"
Sep 11 17:57:57.984748 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.984700    2582 scope.go:117] "RemoveContainer" containerID="e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61"
Sep 11 17:57:57.992933 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.992906    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"]
Sep 11 17:57:57.993430 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.993413    2582 scope.go:117] "RemoveContainer" containerID="e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61"
Sep 11 17:57:57.994184 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:57:57.994151    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61\": container with ID starting with e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61 not found: ID does not exist" containerID="e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61"
Sep 11 17:57:57.994285 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.994191    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61"} err="failed to get container status \"e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61\": rpc error: code = NotFound desc = could not find container \"e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61\": container with ID starting with e09bd01f5fe28eb2e56331d120eefc4b8274cc2807d8eea2b4596fca6a716e61 not found: ID does not exist"
Sep 11 17:57:57.996401 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:57.996334    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-715253-67b5f999d6-pqjnp"]
Sep 11 17:57:58.009329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:58.009305    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"]
Sep 11 17:57:58.011235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:58.011216    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-715253-5d66c584cf-jgbwn"]
Sep 11 17:57:58.502876 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:58.502842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:59.205728 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.205694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:57:59.747730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.747694    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3b883d7d-6e20-4682-a7ee-dde9a2bb9267" path="/var/lib/kubelet/pods/3b883d7d-6e20-4682-a7ee-dde9a2bb9267/volumes"
Sep 11 17:57:59.748177 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.748162    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d827cffe-2253-499c-ad89-6334adeb302b" path="/var/lib/kubelet/pods/d827cffe-2253-499c-ad89-6334adeb302b/volumes"
Sep 11 17:57:59.990763 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.990734    2582 generic.go:356] "Generic (PLEG): container finished" podID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerID="2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e" exitCode=0
Sep 11 17:57:59.990922 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.990821    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" event={"ID":"18de25a0-645d-45eb-8cfa-a4ce4f7471e4","Type":"ContainerDied","Data":"2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"}
Sep 11 17:57:59.991255 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.991238    2582 scope.go:117] "RemoveContainer" containerID="2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"
Sep 11 17:57:59.992471 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.992440    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:57:59.992547 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.992516    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:57:59.992588 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:57:59.992550    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:00.112132 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:00.112099    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:00.876419 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:00.876388    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:01.005025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.004983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" event={"ID":"18de25a0-645d-45eb-8cfa-a4ce4f7471e4","Type":"ContainerStarted","Data":"bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09"}
Sep 11 17:58:01.240530 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.240492    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:58:01.240895 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.240866    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 17:58:01.240998 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.240925    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 17:58:01.288024 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.287988    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6" podUID="e116ab66-1575-4ac6-9bdb-63f1468311ad" containerName="main" containerID="cri-o://3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec" gracePeriod=45
Sep 11 17:58:01.597513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.597486    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:58:01.704224 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704190    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-tmp-dir\") pod \"e116ab66-1575-4ac6-9bdb-63f1468311ad\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") "
Sep 11 17:58:01.704224 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704230    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-dshm\") pod \"e116ab66-1575-4ac6-9bdb-63f1468311ad\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") "
Sep 11 17:58:01.704514 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7plgd\" (UniqueName: \"kubernetes.io/projected/e116ab66-1575-4ac6-9bdb-63f1468311ad-kube-api-access-7plgd\") pod \"e116ab66-1575-4ac6-9bdb-63f1468311ad\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") "
Sep 11 17:58:01.704514 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704290    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-model-cache\") pod \"e116ab66-1575-4ac6-9bdb-63f1468311ad\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") "
Sep 11 17:58:01.704514 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704340    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e116ab66-1575-4ac6-9bdb-63f1468311ad-tls-certs\") pod \"e116ab66-1575-4ac6-9bdb-63f1468311ad\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") "
Sep 11 17:58:01.704709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704588    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-home\") pod \"e116ab66-1575-4ac6-9bdb-63f1468311ad\" (UID: \"e116ab66-1575-4ac6-9bdb-63f1468311ad\") "
Sep 11 17:58:01.704709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704521    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e116ab66-1575-4ac6-9bdb-63f1468311ad" (UID: "e116ab66-1575-4ac6-9bdb-63f1468311ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:01.704709 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704611    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "e116ab66-1575-4ac6-9bdb-63f1468311ad" (UID: "e116ab66-1575-4ac6-9bdb-63f1468311ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:01.704872 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704843    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-home" (OuterVolumeSpecName: "home") pod "e116ab66-1575-4ac6-9bdb-63f1468311ad" (UID: "e116ab66-1575-4ac6-9bdb-63f1468311ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:01.704993 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704972    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:01.705058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.704994    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:01.705058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.705003    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:01.706655 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.706616    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-dshm" (OuterVolumeSpecName: "dshm") pod "e116ab66-1575-4ac6-9bdb-63f1468311ad" (UID: "e116ab66-1575-4ac6-9bdb-63f1468311ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:01.706759 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.706707    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e116ab66-1575-4ac6-9bdb-63f1468311ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e116ab66-1575-4ac6-9bdb-63f1468311ad" (UID: "e116ab66-1575-4ac6-9bdb-63f1468311ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:58:01.706759 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.706734    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e116ab66-1575-4ac6-9bdb-63f1468311ad-kube-api-access-7plgd" (OuterVolumeSpecName: "kube-api-access-7plgd") pod "e116ab66-1575-4ac6-9bdb-63f1468311ad" (UID: "e116ab66-1575-4ac6-9bdb-63f1468311ad"). InnerVolumeSpecName "kube-api-access-7plgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:01.771546 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.771511    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:01.805930 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.805905    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e116ab66-1575-4ac6-9bdb-63f1468311ad-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:01.805930 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.805927    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7plgd\" (UniqueName: \"kubernetes.io/projected/e116ab66-1575-4ac6-9bdb-63f1468311ad-kube-api-access-7plgd\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:01.806090 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:01.805937    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e116ab66-1575-4ac6-9bdb-63f1468311ad-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:02.011785 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.011755    2582 generic.go:356] "Generic (PLEG): container finished" podID="e116ab66-1575-4ac6-9bdb-63f1468311ad" containerID="3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec" exitCode=0
Sep 11 17:58:02.012219 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.011802    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"
Sep 11 17:58:02.012219 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.011841    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6" event={"ID":"e116ab66-1575-4ac6-9bdb-63f1468311ad","Type":"ContainerDied","Data":"3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec"}
Sep 11 17:58:02.012219 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.011889    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6" event={"ID":"e116ab66-1575-4ac6-9bdb-63f1468311ad","Type":"ContainerDied","Data":"fb423df5709940761d68a9dfea1369d9be33d46f0c16c9571ee42fd732e8f04c"}
Sep 11 17:58:02.012219 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.011911    2582 scope.go:117] "RemoveContainer" containerID="3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec"
Sep 11 17:58:02.020969 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.020953    2582 scope.go:117] "RemoveContainer" containerID="3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec"
Sep 11 17:58:02.021200 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:58:02.021179    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec\": container with ID starting with 3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec not found: ID does not exist" containerID="3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec"
Sep 11 17:58:02.021267 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.021212    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec"} err="failed to get container status \"3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec\": rpc error: code = NotFound desc = could not find container \"3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec\": container with ID starting with 3521b6bfd82e9fbd2b011211e4d867c1ce31619b16ad3598b983c6e5f4ee85ec not found: ID does not exist"
Sep 11 17:58:02.035373 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.035338    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"]
Sep 11 17:58:02.040167 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.040138    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-9aab53/test-model-9aab53-kserve-cf46cc79f-nfvz6"]
Sep 11 17:58:02.241269 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.241226    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 17:58:02.241515 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.241295    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 17:58:02.534045 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.534012    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:02.810072 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.809981    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"]
Sep 11 17:58:02.810671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810650    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d827cffe-2253-499c-ad89-6334adeb302b" containerName="payload-processing"
Sep 11 17:58:02.810671 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810670    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d827cffe-2253-499c-ad89-6334adeb302b" containerName="payload-processing"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810687    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b60dd994-82c5-440c-9959-167f1d96b4c9" containerName="revoke-keys"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810695    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b60dd994-82c5-440c-9959-167f1d96b4c9" containerName="revoke-keys"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810712    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e116ab66-1575-4ac6-9bdb-63f1468311ad" containerName="main"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810717    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e116ab66-1575-4ac6-9bdb-63f1468311ad" containerName="main"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810728    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3b883d7d-6e20-4682-a7ee-dde9a2bb9267" containerName="payload-pre-processing"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810736    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3b883d7d-6e20-4682-a7ee-dde9a2bb9267" containerName="payload-pre-processing"
Sep 11 17:58:02.810845 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810839    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3b883d7d-6e20-4682-a7ee-dde9a2bb9267" containerName="payload-pre-processing"
Sep 11 17:58:02.811195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810856    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e116ab66-1575-4ac6-9bdb-63f1468311ad" containerName="main"
Sep 11 17:58:02.811195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810867    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b60dd994-82c5-440c-9959-167f1d96b4c9" containerName="revoke-keys"
Sep 11 17:58:02.811195 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.810877    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d827cffe-2253-499c-ad89-6334adeb302b" containerName="payload-processing"
Sep 11 17:58:02.815232 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.815211    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.817613 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.817595    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ed18394d-openshift-default-dockercfg-ql7hb\""
Sep 11 17:58:02.833708 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.833688    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"]
Sep 11 17:58:02.868372 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.857939    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:02.915578 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.915542    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m28l9\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-kube-api-access-m28l9\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.915884 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.915856    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-credential-socket\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916003 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.915897    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b07338-2128-4d26-a72f-364a10c86a01-istio-podinfo\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916003 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.915975    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b07338-2128-4d26-a72f-364a10c86a01-istiod-ca-cert\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.916003    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-data\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916094 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.916042    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-istio-token\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916188 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.916153    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-socket\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916247 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.916190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-certs\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.916307 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.916276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-envoy\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:02.999126 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:02.999086    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:03.017851 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.017821    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m28l9\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-kube-api-access-m28l9\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.017877    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-credential-socket\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.017981    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b07338-2128-4d26-a72f-364a10c86a01-istio-podinfo\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018046    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b07338-2128-4d26-a72f-364a10c86a01-istiod-ca-cert\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018080    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-data\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018124    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-istio-token\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-socket\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018227    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-certs\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018247    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-credential-socket\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018301 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018264    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-envoy\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018822 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018399    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-data\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018822 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018616    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-socket\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018822 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018675    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-certs\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.018948 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.018825    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b07338-2128-4d26-a72f-364a10c86a01-istiod-ca-cert\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.020958 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.020933    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-envoy\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.021082 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.021066    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b07338-2128-4d26-a72f-364a10c86a01-istio-podinfo\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.027192 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.027169    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m28l9\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-kube-api-access-m28l9\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.027343 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.027324    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-istio-token\") pod \"e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") " pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.126142 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.126061    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:03.251373 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.241705    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 17:58:03.251373 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.241766    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 17:58:03.424879 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.424854    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"]
Sep 11 17:58:03.426788 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:58:03.426755    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7b07338_2128_4d26_a72f_364a10c86a01.slice/crio-d8ffd32bbeaf8cbcc6c42dc74ec113b13f7c9f41b0d1320b0fb38407bd54352d WatchSource:0}: Error finding container d8ffd32bbeaf8cbcc6c42dc74ec113b13f7c9f41b0d1320b0fb38407bd54352d: Status 404 returned error can't find the container with id d8ffd32bbeaf8cbcc6c42dc74ec113b13f7c9f41b0d1320b0fb38407bd54352d
Sep 11 17:58:03.429131 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.429100    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:03.429229 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.429158    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:03.429229 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.429188    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:03.756640 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:03.756597    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e116ab66-1575-4ac6-9bdb-63f1468311ad" path="/var/lib/kubelet/pods/e116ab66-1575-4ac6-9bdb-63f1468311ad/volumes"
Sep 11 17:58:04.027125 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.027032    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" event={"ID":"c7b07338-2128-4d26-a72f-364a10c86a01","Type":"ContainerStarted","Data":"f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"}
Sep 11 17:58:04.027125 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.027098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" event={"ID":"c7b07338-2128-4d26-a72f-364a10c86a01","Type":"ContainerStarted","Data":"d8ffd32bbeaf8cbcc6c42dc74ec113b13f7c9f41b0d1320b0fb38407bd54352d"}
Sep 11 17:58:04.052754 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.052691    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podStartSLOduration=2.052669968 podStartE2EDuration="2.052669968s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:04.050673815 +0000 UTC m=+1166.920716762" watchObservedRunningTime="2026-09-11 17:58:04.052669968 +0000 UTC m=+1166.922712913"
Sep 11 17:58:04.127235 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.127204    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:04.128998 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.128964    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 11 17:58:04.129134 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.129013    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 11 17:58:04.241481 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.241448    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 17:58:04.241729 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.241700    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 17:58:04.314176 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.314066    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:04.384366 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.384320    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:04.581535 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.581432    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:04.930996 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.930907    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"]
Sep 11 17:58:04.937067 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.937036    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:04.949376 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.945703    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 17:58:04.950377 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:04.950017    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"]
Sep 11 17:58:05.053372 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.047916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e76cd-1cc5-48a3-87e9-902918d39097-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr\" (UID: \"ed1e76cd-1cc5-48a3-87e9-902918d39097\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:05.131460 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.131431    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:05.149089 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.149049    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e76cd-1cc5-48a3-87e9-902918d39097-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr\" (UID: \"ed1e76cd-1cc5-48a3-87e9-902918d39097\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:05.149913 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.149882    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e76cd-1cc5-48a3-87e9-902918d39097-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr\" (UID: \"ed1e76cd-1cc5-48a3-87e9-902918d39097\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:05.240871 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.240828    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 11 17:58:05.241026 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.240897    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 11 17:58:05.264916 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.264886    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:05.540455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.540415    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"]
Sep 11 17:58:05.542077 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:58:05.542035    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded1e76cd_1cc5_48a3_87e9_902918d39097.slice/crio-b51844f23563546abd13fd180c9b26f2dd811c1e8a0da8d73ce4c9e557edd2ee WatchSource:0}: Error finding container b51844f23563546abd13fd180c9b26f2dd811c1e8a0da8d73ce4c9e557edd2ee: Status 404 returned error can't find the container with id b51844f23563546abd13fd180c9b26f2dd811c1e8a0da8d73ce4c9e557edd2ee
Sep 11 17:58:05.809731 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:05.809639    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:06.042698 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.042655    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr" event={"ID":"ed1e76cd-1cc5-48a3-87e9-902918d39097","Type":"ContainerStarted","Data":"28f7612c1ca0f89e6bc618f337467eba5b8bfda270c40bfbcca15d1d1ed2f718"}
Sep 11 17:58:06.042907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.042727    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr" event={"ID":"ed1e76cd-1cc5-48a3-87e9-902918d39097","Type":"ContainerStarted","Data":"b51844f23563546abd13fd180c9b26f2dd811c1e8a0da8d73ce4c9e557edd2ee"}
Sep 11 17:58:06.043041 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.043021    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:06.044025 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.044003    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:06.059527 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.059477    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr" podStartSLOduration=2.059461117 podStartE2EDuration="2.059461117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:06.058246138 +0000 UTC m=+1168.928289083" watchObservedRunningTime="2026-09-11 17:58:06.059461117 +0000 UTC m=+1168.929504059"
Sep 11 17:58:06.244968 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.244939    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:58:06.245215 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.245197    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:58:06.245829 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.245813    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 17:58:06.399896 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.399865    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:06.495749 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.495661    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:06.856095 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:06.856010    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:07.050514 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:07.050469    2582 generic.go:356] "Generic (PLEG): container finished" podID="ed1e76cd-1cc5-48a3-87e9-902918d39097" containerID="28f7612c1ca0f89e6bc618f337467eba5b8bfda270c40bfbcca15d1d1ed2f718" exitCode=0
Sep 11 17:58:07.050697 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:07.050537    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr" event={"ID":"ed1e76cd-1cc5-48a3-87e9-902918d39097","Type":"ContainerDied","Data":"28f7612c1ca0f89e6bc618f337467eba5b8bfda270c40bfbcca15d1d1ed2f718"}
Sep 11 17:58:07.741377 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:07.741316    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:08.202003 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:08.201978    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:08.292604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:08.286400    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e76cd-1cc5-48a3-87e9-902918d39097-maas-api-service-ca\") pod \"ed1e76cd-1cc5-48a3-87e9-902918d39097\" (UID: \"ed1e76cd-1cc5-48a3-87e9-902918d39097\") "
Sep 11 17:58:08.292604 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:08.287420    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed1e76cd-1cc5-48a3-87e9-902918d39097-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed1e76cd-1cc5-48a3-87e9-902918d39097" (UID: "ed1e76cd-1cc5-48a3-87e9-902918d39097"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:58:08.388433 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:08.388333    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed1e76cd-1cc5-48a3-87e9-902918d39097-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:08.587030 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:08.586989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:09.063771 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:09.063746    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"
Sep 11 17:58:09.064135 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:09.063779    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr" event={"ID":"ed1e76cd-1cc5-48a3-87e9-902918d39097","Type":"ContainerDied","Data":"b51844f23563546abd13fd180c9b26f2dd811c1e8a0da8d73ce4c9e557edd2ee"}
Sep 11 17:58:09.064135 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:09.063820    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b51844f23563546abd13fd180c9b26f2dd811c1e8a0da8d73ce4c9e557edd2ee"
Sep 11 17:58:09.250830 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:09.250783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:10.126649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:10.126602    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:10.227636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:10.227592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:10.327461 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:10.327430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:10.532617 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:10.532576    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:10.632454 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:10.632417    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:11.073244 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:11.073211    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:11.860776 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:11.860740    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:12.681503 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:12.681464    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:13.470605 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:13.470574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:14.606864 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:14.606830    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:14.897172 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:14.897087    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:15.065338 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:15.065309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:15.307234 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:15.307200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:16.269752 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:16.269718    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:16.464708 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:16.464675    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:16.785638 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:16.785605    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:16.853182 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:16.853152    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"]
Sep 11 17:58:16.857906 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:16.857883    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-9aab53-mfpsr"]
Sep 11 17:58:16.943228 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:16.943194    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:17.262870 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:17.262841    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:17.360999 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:17.360963    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:17.747539 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:17.747504    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed1e76cd-1cc5-48a3-87e9-902918d39097" path="/var/lib/kubelet/pods/ed1e76cd-1cc5-48a3-87e9-902918d39097/volumes"
Sep 11 17:58:18.189643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:18.189559    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:19.140925 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:19.140895    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:19.948305 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:19.948272    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"]
Sep 11 17:58:20.051119 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:20.051082    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:20.182800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:20.182768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:20.904151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:20.904117    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:21.113624 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:21.113577    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:21.470726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:21.470683    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:21.571248 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:21.571219    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:21.871040 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:21.870998    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:22.770458 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:22.770410    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:23.670733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:23.670701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:24.153232 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:24.153201    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:24.253944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:24.253910    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:24.447870 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:24.447779    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:24.510909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:24.510880    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"]
Sep 11 17:58:24.511099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:24.511079    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" containerID="cri-o://e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167" gracePeriod=30
Sep 11 17:58:24.541715 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:24.541684    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:25.132045 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:25.132013    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:25.410214 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:25.410111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:25.731179 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:25.731149    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:27.685928 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:27.685895    2582 patch_prober.go:28] interesting pod/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 17:58:27.686308 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:27.685951    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 17:58:27.947578 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:27.947485    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:28.306388 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:28.306343    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:58:28.616173 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:28.616087    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:28.710826 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:28.710789    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:28.974721 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:28.974641    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:29.670852 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.670821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:29.782367 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.782287    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:58:29.897537 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897509    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-istio-token\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897548    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-socket\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897568    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vn55\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-kube-api-access-9vn55\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897593    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-credential-socket\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897619    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906f1033-bc85-4040-b996-c05975bc35db-istio-podinfo\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897648    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-certs\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897750    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-envoy\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-data\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897817    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906f1033-bc85-4040-b996-c05975bc35db-istiod-ca-cert\") pod \"906f1033-bc85-4040-b996-c05975bc35db\" (UID: \"906f1033-bc85-4040-b996-c05975bc35db\") "
Sep 11 17:58:29.897971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897877    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:29.897971 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.897894    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:29.898253 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.898114    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-data" (OuterVolumeSpecName: "istio-data") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:29.898253 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.898176    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:29.898253 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.898187    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/906f1033-bc85-4040-b996-c05975bc35db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 17:58:29.898450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.898259    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.898450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.898281    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.898450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.898296    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.899907 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.899883    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-istio-token" (OuterVolumeSpecName: "istio-token") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:29.900145 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.900107    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/906f1033-bc85-4040-b996-c05975bc35db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 17:58:29.900230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.900179    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-kube-api-access-9vn55" (OuterVolumeSpecName: "kube-api-access-9vn55") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "kube-api-access-9vn55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:29.900230 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.900178    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "906f1033-bc85-4040-b996-c05975bc35db" (UID: "906f1033-bc85-4040-b996-c05975bc35db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:29.999287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.999245    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.999287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.999279    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vn55\" (UniqueName: \"kubernetes.io/projected/906f1033-bc85-4040-b996-c05975bc35db-kube-api-access-9vn55\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.999287 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.999295    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/906f1033-bc85-4040-b996-c05975bc35db-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.999571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.999307    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.999571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.999318    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/906f1033-bc85-4040-b996-c05975bc35db-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:29.999571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:29.999329    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/906f1033-bc85-4040-b996-c05975bc35db-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:30.192619 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.192531    2582 generic.go:356] "Generic (PLEG): container finished" podID="906f1033-bc85-4040-b996-c05975bc35db" containerID="e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167" exitCode=0
Sep 11 17:58:30.192619 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.192596    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" event={"ID":"906f1033-bc85-4040-b996-c05975bc35db","Type":"ContainerDied","Data":"e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167"}
Sep 11 17:58:30.192841 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.192612    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"
Sep 11 17:58:30.192841 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.192674    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w" event={"ID":"906f1033-bc85-4040-b996-c05975bc35db","Type":"ContainerDied","Data":"f4e81b8052a5a895e6f368657360835d38eb9c8715f4e20602c530ec1c334fc5"}
Sep 11 17:58:30.192841 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.192708    2582 scope.go:117] "RemoveContainer" containerID="e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167"
Sep 11 17:58:30.204641 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.204622    2582 scope.go:117] "RemoveContainer" containerID="9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"
Sep 11 17:58:30.213380 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.213333    2582 scope.go:117] "RemoveContainer" containerID="e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167"
Sep 11 17:58:30.213631 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:58:30.213612    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167\": container with ID starting with e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167 not found: ID does not exist" containerID="e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167"
Sep 11 17:58:30.213684 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.213639    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167"} err="failed to get container status \"e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167\": rpc error: code = NotFound desc = could not find container \"e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167\": container with ID starting with e346731c85df778b7b450ffb9f0741f6e6dcb3808e79ae03cb8220de65217167 not found: ID does not exist"
Sep 11 17:58:30.213684 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.213655    2582 scope.go:117] "RemoveContainer" containerID="9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"
Sep 11 17:58:30.213904 ip-10-0-128-237 kubenswrapper[2582]: E0911 17:58:30.213889    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc\": container with ID starting with 9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc not found: ID does not exist" containerID="9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"
Sep 11 17:58:30.213945 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.213907    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc"} err="failed to get container status \"9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc\": rpc error: code = NotFound desc = could not find container \"9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc\": container with ID starting with 9a1e2730f6b5489fabcb4be0fb8d09909e37475706617f99402d77b18bb877bc not found: ID does not exist"
Sep 11 17:58:30.218269 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.218249    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"]
Sep 11 17:58:30.224482 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.224461    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9aab53-openshift-default-59d44fc57d-cnf4w"]
Sep 11 17:58:30.608511 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:30.608477    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:31.357751 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:31.357711    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:31.752151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:31.752107    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906f1033-bc85-4040-b996-c05975bc35db" path="/var/lib/kubelet/pods/906f1033-bc85-4040-b996-c05975bc35db/volumes"
Sep 11 17:58:32.328058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:32.328020    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:33.264965 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:33.264928    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:33.392791 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:33.392750    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:33.798024 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:33.797977    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:33.999856 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:33.999822    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:34.558288 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:34.558257    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:34.967882 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:34.967789    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf" podUID="4295924b-0b61-4a08-8994-9629b048fd2f" containerName="main" containerID="cri-o://66306600fb2ca5eaba1baf46c7e852ef2d73386f6969a6c6be1191bad111e34a" gracePeriod=45
Sep 11 17:58:35.225769 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.225738    2582 generic.go:356] "Generic (PLEG): container finished" podID="4295924b-0b61-4a08-8994-9629b048fd2f" containerID="66306600fb2ca5eaba1baf46c7e852ef2d73386f6969a6c6be1191bad111e34a" exitCode=0
Sep 11 17:58:35.225867 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.225797    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf" event={"ID":"4295924b-0b61-4a08-8994-9629b048fd2f","Type":"ContainerDied","Data":"66306600fb2ca5eaba1baf46c7e852ef2d73386f6969a6c6be1191bad111e34a"}
Sep 11 17:58:35.225867 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.225828    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf" event={"ID":"4295924b-0b61-4a08-8994-9629b048fd2f","Type":"ContainerDied","Data":"b4b8052b95364a8ba9998cc54dd96191bf946a8c9f2da9e9b9c1687edcd76bc6"}
Sep 11 17:58:35.225867 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.225843    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4b8052b95364a8ba9998cc54dd96191bf946a8c9f2da9e9b9c1687edcd76bc6"
Sep 11 17:58:35.270541 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.270517    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:58:35.366371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.365806    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-tmp-dir\") pod \"4295924b-0b61-4a08-8994-9629b048fd2f\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") "
Sep 11 17:58:35.366371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.365871    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-model-cache\") pod \"4295924b-0b61-4a08-8994-9629b048fd2f\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") "
Sep 11 17:58:35.366371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.365906    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-dshm\") pod \"4295924b-0b61-4a08-8994-9629b048fd2f\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") "
Sep 11 17:58:35.366371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.365935    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4295924b-0b61-4a08-8994-9629b048fd2f-tls-certs\") pod \"4295924b-0b61-4a08-8994-9629b048fd2f\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") "
Sep 11 17:58:35.366371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.365964    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnmf6\" (UniqueName: \"kubernetes.io/projected/4295924b-0b61-4a08-8994-9629b048fd2f-kube-api-access-xnmf6\") pod \"4295924b-0b61-4a08-8994-9629b048fd2f\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") "
Sep 11 17:58:35.366898 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.366868    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-model-cache" (OuterVolumeSpecName: "model-cache") pod "4295924b-0b61-4a08-8994-9629b048fd2f" (UID: "4295924b-0b61-4a08-8994-9629b048fd2f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:35.367087 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.367059    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4295924b-0b61-4a08-8994-9629b048fd2f" (UID: "4295924b-0b61-4a08-8994-9629b048fd2f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:35.370140 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.370011    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4295924b-0b61-4a08-8994-9629b048fd2f-kube-api-access-xnmf6" (OuterVolumeSpecName: "kube-api-access-xnmf6") pod "4295924b-0b61-4a08-8994-9629b048fd2f" (UID: "4295924b-0b61-4a08-8994-9629b048fd2f"). InnerVolumeSpecName "kube-api-access-xnmf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 17:58:35.370416 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.370388    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-dshm" (OuterVolumeSpecName: "dshm") pod "4295924b-0b61-4a08-8994-9629b048fd2f" (UID: "4295924b-0b61-4a08-8994-9629b048fd2f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:35.370704 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.370682    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4295924b-0b61-4a08-8994-9629b048fd2f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4295924b-0b61-4a08-8994-9629b048fd2f" (UID: "4295924b-0b61-4a08-8994-9629b048fd2f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 17:58:35.467171 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467135    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-home\") pod \"4295924b-0b61-4a08-8994-9629b048fd2f\" (UID: \"4295924b-0b61-4a08-8994-9629b048fd2f\") "
Sep 11 17:58:35.467400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467342    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:35.467400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467372    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:35.467400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467379    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:35.467400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467388    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4295924b-0b61-4a08-8994-9629b048fd2f-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:35.467400 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467401    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnmf6\" (UniqueName: \"kubernetes.io/projected/4295924b-0b61-4a08-8994-9629b048fd2f-kube-api-access-xnmf6\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:35.467673 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.467601    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-home" (OuterVolumeSpecName: "home") pod "4295924b-0b61-4a08-8994-9629b048fd2f" (UID: "4295924b-0b61-4a08-8994-9629b048fd2f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 17:58:35.568451 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.568410    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4295924b-0b61-4a08-8994-9629b048fd2f-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 17:58:35.773341 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:35.773306    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:36.231919 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:36.231887    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"
Sep 11 17:58:36.272734 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:36.272703    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"]
Sep 11 17:58:36.280856 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:36.280826    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-6baa6b/noten-model-6baa6b-kserve-5766cbf59f-r7gwf"]
Sep 11 17:58:37.084545 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.084504    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 17:58:37.084933 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.084580    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 17:58:37.657775 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.657749    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:58:37.659744 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.659721    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 17:58:37.720659 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.720628    2582 scope.go:117] "RemoveContainer" containerID="ac801f28df95a4a6bdcae7452a65451cf7630140392d5f86f867485feef4851a"
Sep 11 17:58:37.730777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.730754    2582 scope.go:117] "RemoveContainer" containerID="f6f0a72f8616057e0eeb249958d3b16a57810640dcc3a868bfd858d797274844"
Sep 11 17:58:37.757639 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:37.757534    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4295924b-0b61-4a08-8994-9629b048fd2f" path="/var/lib/kubelet/pods/4295924b-0b61-4a08-8994-9629b048fd2f/volumes"
Sep 11 17:58:38.078644 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.078609    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"]
Sep 11 17:58:38.079143 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079119    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4295924b-0b61-4a08-8994-9629b048fd2f" containerName="main"
Sep 11 17:58:38.079143 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079140    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4295924b-0b61-4a08-8994-9629b048fd2f" containerName="main"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079160    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed1e76cd-1cc5-48a3-87e9-902918d39097" containerName="revoke-keys"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079169    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed1e76cd-1cc5-48a3-87e9-902918d39097" containerName="revoke-keys"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079180    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079188    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079200    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079206    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079303    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079314    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed1e76cd-1cc5-48a3-87e9-902918d39097" containerName="revoke-keys"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079320    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="906f1033-bc85-4040-b996-c05975bc35db" containerName="istio-proxy"
Sep 11 17:58:38.079340 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.079331    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4295924b-0b61-4a08-8994-9629b048fd2f" containerName="main"
Sep 11 17:58:38.084450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.084422    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.087448 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.087429    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"default-dockercfg-n6lmt\""
Sep 11 17:58:38.087448 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.087438    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"openshift-service-ca.crt\""
Sep 11 17:58:38.087800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.087438    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"distinct-2-w0-286c27-kserve-self-signed-certs\""
Sep 11 17:58:38.087800 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.087447    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-286c27\"/\"kube-root-ca.crt\""
Sep 11 17:58:38.099652 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.099626    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"]
Sep 11 17:58:38.195310 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.195270    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-model-cache\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.195513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.195323    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5lgh\" (UniqueName: \"kubernetes.io/projected/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-kube-api-access-s5lgh\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.195513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.195414    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-home\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.195513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.195444    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tls-certs\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.195513 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.195487    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tmp-dir\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.195702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.195556    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-dshm\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296034 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-dshm\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296046    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-model-cache\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5lgh\" (UniqueName: \"kubernetes.io/projected/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-kube-api-access-s5lgh\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296110    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-home\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tls-certs\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296201 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296180    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tmp-dir\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296582 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296555    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-model-cache\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296662 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296565    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tmp-dir\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.296662 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.296623    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-home\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.298196 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.298174    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-dshm\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.298673 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.298654    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tls-certs\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.305472 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.305454    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5lgh\" (UniqueName: \"kubernetes.io/projected/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-kube-api-access-s5lgh\") pod \"distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") " pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.310681 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.310652    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:38.397324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.397240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:38.671374 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:38.671333    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"]
Sep 11 17:58:38.674326 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:58:38.674293    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddbb319f5_b5b2_4cdb_81bc_9332bbcf1ade.slice/crio-e0a56259fea8f3a9d0832742fabadd212e652d2103249ae196aea767d976a0a4 WatchSource:0}: Error finding container e0a56259fea8f3a9d0832742fabadd212e652d2103249ae196aea767d976a0a4: Status 404 returned error can't find the container with id e0a56259fea8f3a9d0832742fabadd212e652d2103249ae196aea767d976a0a4
Sep 11 17:58:39.253567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.253485    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p" event={"ID":"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade","Type":"ContainerStarted","Data":"4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9"}
Sep 11 17:58:39.253567 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.253530    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p" event={"ID":"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade","Type":"ContainerStarted","Data":"e0a56259fea8f3a9d0832742fabadd212e652d2103249ae196aea767d976a0a4"}
Sep 11 17:58:39.255040 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.255015    2582 generic.go:356] "Generic (PLEG): container finished" podID="c7b07338-2128-4d26-a72f-364a10c86a01" containerID="f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89" exitCode=0
Sep 11 17:58:39.255155 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.255087    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" event={"ID":"c7b07338-2128-4d26-a72f-364a10c86a01","Type":"ContainerDied","Data":"f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"}
Sep 11 17:58:39.255492 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.255478    2582 scope.go:117] "RemoveContainer" containerID="f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"
Sep 11 17:58:39.256745 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.256717    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:39.256817 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.256782    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:39.256817 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.256812    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:39.273975 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:39.273935    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p" podStartSLOduration=1.273920645 podStartE2EDuration="1.273920645s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:58:39.271828851 +0000 UTC m=+1202.141871796" watchObservedRunningTime="2026-09-11 17:58:39.273920645 +0000 UTC m=+1202.143963590"
Sep 11 17:58:40.267053 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:40.267008    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" event={"ID":"c7b07338-2128-4d26-a72f-364a10c86a01","Type":"ContainerStarted","Data":"45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9"}
Sep 11 17:58:41.126246 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:41.126207    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:41.126450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:41.126385    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 11 17:58:41.126450 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:41.126429    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 11 17:58:42.127084 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:42.127048    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 11 17:58:42.127477 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:42.127104    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 11 17:58:43.127427 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:43.127393    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 11 17:58:43.127804 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:43.127445    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 11 17:58:43.221935 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:43.221899    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:44.127359 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:44.127308    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 11 17:58:44.127515 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:44.127383    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 11 17:58:45.127179 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:45.127145    2582 patch_prober.go:28] interesting pod/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused" start-of-body=
Sep 11 17:58:45.127372 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:45.127201    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.69:15021/healthz/ready\": dial tcp 10.132.0.69:15021: connect: connection refused"
Sep 11 17:58:46.130884 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:46.130850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:46.131252 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:46.131109    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:46.131679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:46.131659    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 17:58:48.397944 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:48.397913    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:48.398488 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:48.397957    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:48.410674 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:48.410649    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:49.330006 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:49.329967    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 17:58:55.983026 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:55.982991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:57.266595 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:57.266564    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:57.507413 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:57.507376    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:57.608069 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:57.607971    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:58.506024 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:58.505991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:58.694444 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:58.694407    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:59.066780 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.066741    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:59.358420 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.358303    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"]
Sep 11 17:58:59.362576 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.362547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:59.362916 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.362899    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.365162 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.365142    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-1bf937-openshift-default-dockercfg-blrhr\""
Sep 11 17:58:59.375478 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.375451    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"]
Sep 11 17:58:59.427937 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.427902    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:59.518908 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.518875    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-certs\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.518916    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-podinfo\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519000    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-token\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istiod-ca-cert\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519063    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-credential-socket\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519147    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-data\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519184    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-socket\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519268 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519208    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-envoy\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.519540 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.519270    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znbzr\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-kube-api-access-znbzr\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.528329 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.528295    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:58:59.620636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-socket\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620588    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-envoy\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620636 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znbzr\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-kube-api-access-znbzr\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620686    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-certs\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620717    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-podinfo\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620751    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-token\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620769    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istiod-ca-cert\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620805    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-credential-socket\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.620929 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.620850    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-data\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.621242 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.621146    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-certs\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.621242 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.621198    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-credential-socket\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.621383 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.621258    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-socket\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.621383 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.621283    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-data\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.621550 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.621529    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istiod-ca-cert\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.622986 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.622957    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-envoy\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.623191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.623176    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-podinfo\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.629499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.629478    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-token\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.629816 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.629794    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znbzr\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-kube-api-access-znbzr\") pod \"e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") " pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.678862 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.678834    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:58:59.955798 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.955769    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"]
Sep 11 17:58:59.957817 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:58:59.957795    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9dea0c_e3c3_4e35_929d_fa5ec0ccad58.slice/crio-3a33b920e878cb915602dd9c0e6603e6ef7d194c969c3a2c524fef8ef7c5ee9b WatchSource:0}: Error finding container 3a33b920e878cb915602dd9c0e6603e6ef7d194c969c3a2c524fef8ef7c5ee9b: Status 404 returned error can't find the container with id 3a33b920e878cb915602dd9c0e6603e6ef7d194c969c3a2c524fef8ef7c5ee9b
Sep 11 17:58:59.960581 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.960548    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:59.960670 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.960610    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:58:59.960670 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:58:59.960635    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:59:00.077134 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:00.077092    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:00.383608 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:00.383573    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" event={"ID":"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58","Type":"ContainerStarted","Data":"39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"}
Sep 11 17:59:00.383752 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:00.383654    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" event={"ID":"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58","Type":"ContainerStarted","Data":"3a33b920e878cb915602dd9c0e6603e6ef7d194c969c3a2c524fef8ef7c5ee9b"}
Sep 11 17:59:00.417026 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:00.416973    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podStartSLOduration=1.416957628 podStartE2EDuration="1.416957628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:58:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:59:00.415899059 +0000 UTC m=+1223.285942005" watchObservedRunningTime="2026-09-11 17:59:00.416957628 +0000 UTC m=+1223.287000573"
Sep 11 17:59:00.679690 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:00.679610    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:00.870359 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:00.870315    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:01.684569 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:01.684538    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:01.769516 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:01.769479    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:02.395809 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:02.395779    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:02.396793 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:02.396771    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:02.580576 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:02.580547    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:02.764555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:02.764523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:02.944542 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:02.944508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:03.886702 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:03.886669    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:04.905913 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:04.905879    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:05.788151 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:05.788113    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:06.486455 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:06.486417    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:07.291062 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:07.291031    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:08.081863 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:08.081828    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:08.182723 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:08.182690    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:08.235764 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:08.235722    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:08.655959 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:08.655922    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:08.756499 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:08.756465    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:10.218376 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:10.218331    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:59:20.478726 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.478691    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"]
Sep 11 17:59:20.482524 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.482507    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.485522 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.485501    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ed18394d\"/\"kube-root-ca.crt\""
Sep 11 17:59:20.485661 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.485571    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ed18394d\"/\"e2e-lifecycle-model-ed18394d-kserve-self-signed-certs\""
Sep 11 17:59:20.485661 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.485591    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ed18394d\"/\"openshift-service-ca.crt\""
Sep 11 17:59:20.485661 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.485587    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ed18394d\"/\"default-dockercfg-9lb29\""
Sep 11 17:59:20.492191 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.492170    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"]
Sep 11 17:59:20.529245 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.529211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-dshm\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.529438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.529274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7rw9\" (UniqueName: \"kubernetes.io/projected/1aadb12d-10d1-4533-baf3-c04f55f2219a-kube-api-access-z7rw9\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.529438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.529321    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aadb12d-10d1-4533-baf3-c04f55f2219a-tls-certs\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.529438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.529379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-tmp-dir\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.529438 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.529429    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-home\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.529674 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.529547    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-model-cache\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.569501 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.569460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:20.630058 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-dshm\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630189 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630068    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7rw9\" (UniqueName: \"kubernetes.io/projected/1aadb12d-10d1-4533-baf3-c04f55f2219a-kube-api-access-z7rw9\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630189 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aadb12d-10d1-4533-baf3-c04f55f2219a-tls-certs\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630189 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630119    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-tmp-dir\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630189 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-home\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630451 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630246    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-model-cache\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630638 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630611    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-home\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630616    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-tmp-dir\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.630770 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.630651    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-model-cache\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.632409 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.632388    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-dshm\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.632679 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.632663    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aadb12d-10d1-4533-baf3-c04f55f2219a-tls-certs\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.640117 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.640090    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7rw9\" (UniqueName: \"kubernetes.io/projected/1aadb12d-10d1-4533-baf3-c04f55f2219a-kube-api-access-z7rw9\") pod \"e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") " pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:20.793722 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:20.793688    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:21.077215 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:21.077188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"]
Sep 11 17:59:21.079659 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:59:21.079632    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1aadb12d_10d1_4533_baf3_c04f55f2219a.slice/crio-e7acc7f78151d9dbc33623db1b0066f3c0079f7fcf2576283e7031f431b04646 WatchSource:0}: Error finding container e7acc7f78151d9dbc33623db1b0066f3c0079f7fcf2576283e7031f431b04646: Status 404 returned error can't find the container with id e7acc7f78151d9dbc33623db1b0066f3c0079f7fcf2576283e7031f431b04646
Sep 11 17:59:21.511395 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:21.511363    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq" event={"ID":"1aadb12d-10d1-4533-baf3-c04f55f2219a","Type":"ContainerStarted","Data":"90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c"}
Sep 11 17:59:21.511777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:21.511402    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq" event={"ID":"1aadb12d-10d1-4533-baf3-c04f55f2219a","Type":"ContainerStarted","Data":"e7acc7f78151d9dbc33623db1b0066f3c0079f7fcf2576283e7031f431b04646"}
Sep 11 17:59:21.528817 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:21.528772    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq" podStartSLOduration=1.528754699 podStartE2EDuration="1.528754699s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:59:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:59:21.52733222 +0000 UTC m=+1244.397375164" watchObservedRunningTime="2026-09-11 17:59:21.528754699 +0000 UTC m=+1244.398797645"
Sep 11 17:59:30.461730 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:30.461631    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:30.794371 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:30.794316    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:30.794555 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:30.794389    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:30.807323 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:30.807296    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:31.122075 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.121989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:31.578995 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.578971    2582 generic.go:356] "Generic (PLEG): container finished" podID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerID="39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d" exitCode=0
Sep 11 17:59:31.579324 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.579038    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" event={"ID":"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58","Type":"ContainerDied","Data":"39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"}
Sep 11 17:59:31.579562 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.579544    2582 scope.go:117] "RemoveContainer" containerID="39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"
Sep 11 17:59:31.580820 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.580786    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:59:31.580909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.580859    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:59:31.580909 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.580891    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 17:59:31.590777 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.590757    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 17:59:31.961819 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:31.961737    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:32.396572 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.396540    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:32.519733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.519691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:32.597608 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.597570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" event={"ID":"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58","Type":"ContainerStarted","Data":"4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67"}
Sep 11 17:59:32.679643 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.679558    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:32.680640 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.680611    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 11 17:59:32.680759 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.680661    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 11 17:59:32.719173 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:32.719140    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:33.544571 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:33.544533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:33.679693 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:33.679656    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 11 17:59:33.680099 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:33.679722    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 11 17:59:34.264491 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:34.264453    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:34.680265 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:34.680178    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 11 17:59:34.680265 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:34.680234    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 11 17:59:35.050893 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:35.050859    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:35.679273 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:35.679236    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 11 17:59:35.679456 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:35.679306    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 11 17:59:35.991404 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:35.991373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:36.679468 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:36.679428    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 11 17:59:36.679661 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:36.679487    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 11 17:59:37.679264 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:37.679230    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Sep 11 17:59:37.679649 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:37.679286    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Sep 11 17:59:37.786872 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:37.786841    2582 scope.go:117] "RemoveContainer" containerID="b5ff858317a4143359cd990af6ba1e6787f945dc92f2a09c155c73187db366d0"
Sep 11 17:59:38.123739 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:38.123706    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 17:59:38.705658 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:38.705564    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:38.706207 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:38.705964    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:38.706824 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:38.706806    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 17:59:58.384095 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.384061    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"]
Sep 11 17:59:58.389552 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.389527    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.392160 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.392133    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"openshift-service-ca.crt\""
Sep 11 17:59:58.392665 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.392645    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"auth-test-model-a18ac6-kserve-self-signed-certs\""
Sep 11 17:59:58.392733 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.392702    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"default-dockercfg-q7pct\""
Sep 11 17:59:58.395194 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.395175    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"kube-root-ca.crt\""
Sep 11 17:59:58.401133 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.401107    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"]
Sep 11 17:59:58.423008 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.422978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:58.523544 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.523510    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkznd\" (UniqueName: \"kubernetes.io/projected/e1228f2f-278a-4423-94b5-28beb41ef2a4-kube-api-access-zkznd\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.523716 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.523569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-tmp-dir\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.523716 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.523667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1228f2f-278a-4423-94b5-28beb41ef2a4-tls-certs\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.523716 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.523702    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-dshm\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.523851 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.523732    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-home\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.523851 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.523762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-model-cache\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625307 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625263    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkznd\" (UniqueName: \"kubernetes.io/projected/e1228f2f-278a-4423-94b5-28beb41ef2a4-kube-api-access-zkznd\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625507 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625330    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-tmp-dir\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625578 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625505    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1228f2f-278a-4423-94b5-28beb41ef2a4-tls-certs\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625578 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-dshm\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625692 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625612    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-home\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625692 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-model-cache\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625814 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625788    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-tmp-dir\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.625948 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625919    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-home\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.626032 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.625978    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-model-cache\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.628240 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.628213    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-dshm\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.628518 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.628498    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1228f2f-278a-4423-94b5-28beb41ef2a4-tls-certs\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.633880 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.633859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkznd\" (UniqueName: \"kubernetes.io/projected/e1228f2f-278a-4423-94b5-28beb41ef2a4-kube-api-access-zkznd\") pod \"auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.700884 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.700788    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 17:59:58.980779 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:58.980745    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"]
Sep 11 17:59:58.982536 ip-10-0-128-237 kubenswrapper[2582]: W0911 17:59:58.982505    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1228f2f_278a_4423_94b5_28beb41ef2a4.slice/crio-158567e302586719f15fd3ff8651d2c4c4b7d2fe77437f4807978281c551a438 WatchSource:0}: Error finding container 158567e302586719f15fd3ff8651d2c4c4b7d2fe77437f4807978281c551a438: Status 404 returned error can't find the container with id 158567e302586719f15fd3ff8651d2c4c4b7d2fe77437f4807978281c551a438
Sep 11 17:59:59.139272 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:59.139235    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 17:59:59.767704 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:59.767656    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" event={"ID":"e1228f2f-278a-4423-94b5-28beb41ef2a4","Type":"ContainerStarted","Data":"14335e87942cf05e565f75bb7c05a82e512b7d55899aa5e6e04db87a02d3408e"}
Sep 11 17:59:59.768251 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:59.767714    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" event={"ID":"e1228f2f-278a-4423-94b5-28beb41ef2a4","Type":"ContainerStarted","Data":"158567e302586719f15fd3ff8651d2c4c4b7d2fe77437f4807978281c551a438"}
Sep 11 17:59:59.787985 ip-10-0-128-237 kubenswrapper[2582]: I0911 17:59:59.787933    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" podStartSLOduration=1.787914676 podStartE2EDuration="1.787914676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 17:59:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 17:59:59.78756709 +0000 UTC m=+1282.657610035" watchObservedRunningTime="2026-09-11 17:59:59.787914676 +0000 UTC m=+1282.657957622"
Sep 11 18:00:00.161196 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.161115    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"]
Sep 11 18:00:00.165728 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.165702    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"]
Sep 11 18:00:00.165867 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.165812    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"
Sep 11 18:00:00.168011 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.167985    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:00:00.169573 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.169553    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"]
Sep 11 18:00:00.169712 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.169684    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"
Sep 11 18:00:00.173202 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.173178    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"
Sep 11 18:00:00.176130 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.176058    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"]
Sep 11 18:00:00.180457 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.179522    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"]
Sep 11 18:00:00.181114 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.181098    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"
Sep 11 18:00:00.182650 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.182628    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"]
Sep 11 18:00:00.194943 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.193864    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"
Sep 11 18:00:00.199995 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.199366    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"]
Sep 11 18:00:00.205466 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.205442    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"
Sep 11 18:00:00.216843 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.216818    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"]
Sep 11 18:00:00.217223 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.217203    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"
Sep 11 18:00:00.223779 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.223636    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"]
Sep 11 18:00:00.223779 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.223674    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"]
Sep 11 18:00:00.223779 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.223768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"
Sep 11 18:00:00.287475 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.287295    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"
Sep 11 18:00:00.299052 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.298707    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"
Sep 11 18:00:00.526488 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.526444    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"]
Sep 11 18:00:00.527020 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:00:00.526990    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cc779f_e56c_4316_a8ce_b74132019e8d.slice/crio-d87704cccaadac379f899c8afa51ea6642e7de973c8aed977fd321e3611b8b9c WatchSource:0}: Error finding container d87704cccaadac379f899c8afa51ea6642e7de973c8aed977fd321e3611b8b9c: Status 404 returned error can't find the container with id d87704cccaadac379f899c8afa51ea6642e7de973c8aed977fd321e3611b8b9c
Sep 11 18:00:00.535525 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.535500    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"]
Sep 11 18:00:00.537179 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:00:00.537151    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ebb2753_613c_41d7_b50c_dcc7e6705451.slice/crio-39a0c3a5b4f5fa7d08b9acd0eaeffc93411ca3571291f86782eaecb3e4ca05a6 WatchSource:0}: Error finding container 39a0c3a5b4f5fa7d08b9acd0eaeffc93411ca3571291f86782eaecb3e4ca05a6: Status 404 returned error can't find the container with id 39a0c3a5b4f5fa7d08b9acd0eaeffc93411ca3571291f86782eaecb3e4ca05a6
Sep 11 18:00:00.574403 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.574376    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"]
Sep 11 18:00:00.577224 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:00:00.577184    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44da4928_5a19_40bf_874d_df9d180b51de.slice/crio-8328f733d01b1626745c840ab502a76d7582538a3c9bc8d6e13f19395f4d49d9 WatchSource:0}: Error finding container 8328f733d01b1626745c840ab502a76d7582538a3c9bc8d6e13f19395f4d49d9: Status 404 returned error can't find the container with id 8328f733d01b1626745c840ab502a76d7582538a3c9bc8d6e13f19395f4d49d9
Sep 11 18:00:00.655548 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.655308    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"]
Sep 11 18:00:00.659199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.659174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"]
Sep 11 18:00:00.740871 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:00:00.740826    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b9e412_d3ff_4783_a2da_6ab0eb0a9196.slice/crio-e1993403de9adf0d79080a71c569627ca2d4d4d22a45e947fc5ea833fd25549f WatchSource:0}: Error finding container e1993403de9adf0d79080a71c569627ca2d4d4d22a45e947fc5ea833fd25549f: Status 404 returned error can't find the container with id e1993403de9adf0d79080a71c569627ca2d4d4d22a45e947fc5ea833fd25549f
Sep 11 18:00:00.741835 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:00:00.741792    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2c7574f_9a06_4425_9627_593fc531cb6d.slice/crio-500b57e568b94fa7a2d20565c0b9fd1eb39ff915504ffe2c04cd262fb6edfd9d WatchSource:0}: Error finding container 500b57e568b94fa7a2d20565c0b9fd1eb39ff915504ffe2c04cd262fb6edfd9d: Status 404 returned error can't find the container with id 500b57e568b94fa7a2d20565c0b9fd1eb39ff915504ffe2c04cd262fb6edfd9d
Sep 11 18:00:00.780025 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.779934    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n" event={"ID":"77b9e412-d3ff-4783-a2da-6ab0eb0a9196","Type":"ContainerStarted","Data":"e1993403de9adf0d79080a71c569627ca2d4d4d22a45e947fc5ea833fd25549f"}
Sep 11 18:00:00.782697 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.782615    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2" event={"ID":"e4cc779f-e56c-4316-a8ce-b74132019e8d","Type":"ContainerStarted","Data":"d87704cccaadac379f899c8afa51ea6642e7de973c8aed977fd321e3611b8b9c"}
Sep 11 18:00:00.785411 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.785385    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd" event={"ID":"c2c7574f-9a06-4425-9627-593fc531cb6d","Type":"ContainerStarted","Data":"500b57e568b94fa7a2d20565c0b9fd1eb39ff915504ffe2c04cd262fb6edfd9d"}
Sep 11 18:00:00.787518 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.787499    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt" event={"ID":"44da4928-5a19-40bf-874d-df9d180b51de","Type":"ContainerStarted","Data":"8328f733d01b1626745c840ab502a76d7582538a3c9bc8d6e13f19395f4d49d9"}
Sep 11 18:00:00.789132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:00.789104    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn" event={"ID":"7ebb2753-613c-41d7-b50c-dcc7e6705451","Type":"ContainerStarted","Data":"39a0c3a5b4f5fa7d08b9acd0eaeffc93411ca3571291f86782eaecb3e4ca05a6"}
Sep 11 18:00:01.099563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.099525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:01.797892 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.797852    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2" event={"ID":"e4cc779f-e56c-4316-a8ce-b74132019e8d","Type":"ContainerStarted","Data":"d1e81343ea2be5d00567d96952e3720ec5ef3f923b345e16023e192f19a03fca"}
Sep 11 18:00:01.799649 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.799614    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd" event={"ID":"c2c7574f-9a06-4425-9627-593fc531cb6d","Type":"ContainerStarted","Data":"5c898e937dcec56a3160b4297100bb512c872f10ee21917c0e27be65f167d611"}
Sep 11 18:00:01.801521 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.801445    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt" event={"ID":"44da4928-5a19-40bf-874d-df9d180b51de","Type":"ContainerStarted","Data":"db31b87fe75bcdebd82631aa5af8b2804161d2c4cf27fd77f7bfb873facc9887"}
Sep 11 18:00:01.803618 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.803590    2582 generic.go:356] "Generic (PLEG): container finished" podID="7ebb2753-613c-41d7-b50c-dcc7e6705451" containerID="f629aff52fa018fa3b67c7558acc6fc7c2201716432726abf6eede24e67a65a4" exitCode=0
Sep 11 18:00:01.803758 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.803666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn" event={"ID":"7ebb2753-613c-41d7-b50c-dcc7e6705451","Type":"ContainerDied","Data":"f629aff52fa018fa3b67c7558acc6fc7c2201716432726abf6eede24e67a65a4"}
Sep 11 18:00:01.805616 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.805590    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n" event={"ID":"77b9e412-d3ff-4783-a2da-6ab0eb0a9196","Type":"ContainerStarted","Data":"7cb67dcbedcadd05fe85a9cd669786cbc69813e0f5ab7360374b3e356eb40df7"}
Sep 11 18:00:01.820958 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.820913    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2" podStartSLOduration=1.820894783 podStartE2EDuration="1.820894783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:01.82001483 +0000 UTC m=+1284.690057792" watchObservedRunningTime="2026-09-11 18:00:01.820894783 +0000 UTC m=+1284.690937775"
Sep 11 18:00:01.866410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.865924    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n" podStartSLOduration=1.865903686 podStartE2EDuration="1.865903686s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:01.860533942 +0000 UTC m=+1284.730576878" watchObservedRunningTime="2026-09-11 18:00:01.865903686 +0000 UTC m=+1284.735946633"
Sep 11 18:00:01.877040 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.876990    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd" podStartSLOduration=1.876973634 podStartE2EDuration="1.876973634s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:01.876196458 +0000 UTC m=+1284.746239415" watchObservedRunningTime="2026-09-11 18:00:01.876973634 +0000 UTC m=+1284.747016575"
Sep 11 18:00:01.892712 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:01.892665    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt" podStartSLOduration=1.8926496350000002 podStartE2EDuration="1.892649635s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:00:01.892161791 +0000 UTC m=+1284.762204739" watchObservedRunningTime="2026-09-11 18:00:01.892649635 +0000 UTC m=+1284.762692592"
Sep 11 18:00:02.813853 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.813820    2582 generic.go:356] "Generic (PLEG): container finished" podID="77b9e412-d3ff-4783-a2da-6ab0eb0a9196" containerID="7cb67dcbedcadd05fe85a9cd669786cbc69813e0f5ab7360374b3e356eb40df7" exitCode=0
Sep 11 18:00:02.814327 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.813906    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n" event={"ID":"77b9e412-d3ff-4783-a2da-6ab0eb0a9196","Type":"ContainerDied","Data":"7cb67dcbedcadd05fe85a9cd669786cbc69813e0f5ab7360374b3e356eb40df7"}
Sep 11 18:00:02.815768 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.815591    2582 generic.go:356] "Generic (PLEG): container finished" podID="e4cc779f-e56c-4316-a8ce-b74132019e8d" containerID="d1e81343ea2be5d00567d96952e3720ec5ef3f923b345e16023e192f19a03fca" exitCode=0
Sep 11 18:00:02.815768 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.815714    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2" event={"ID":"e4cc779f-e56c-4316-a8ce-b74132019e8d","Type":"ContainerDied","Data":"d1e81343ea2be5d00567d96952e3720ec5ef3f923b345e16023e192f19a03fca"}
Sep 11 18:00:02.817818 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.817646    2582 generic.go:356] "Generic (PLEG): container finished" podID="c2c7574f-9a06-4425-9627-593fc531cb6d" containerID="5c898e937dcec56a3160b4297100bb512c872f10ee21917c0e27be65f167d611" exitCode=0
Sep 11 18:00:02.817818 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.817745    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd" event={"ID":"c2c7574f-9a06-4425-9627-593fc531cb6d","Type":"ContainerDied","Data":"5c898e937dcec56a3160b4297100bb512c872f10ee21917c0e27be65f167d611"}
Sep 11 18:00:02.823886 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.823861    2582 generic.go:356] "Generic (PLEG): container finished" podID="44da4928-5a19-40bf-874d-df9d180b51de" containerID="db31b87fe75bcdebd82631aa5af8b2804161d2c4cf27fd77f7bfb873facc9887" exitCode=0
Sep 11 18:00:02.824409 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:02.824139    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt" event={"ID":"44da4928-5a19-40bf-874d-df9d180b51de","Type":"ContainerDied","Data":"db31b87fe75bcdebd82631aa5af8b2804161d2c4cf27fd77f7bfb873facc9887"}
Sep 11 18:00:03.022564 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:03.022528    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"
Sep 11 18:00:03.833402 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:03.833371    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"
Sep 11 18:00:03.833402 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:03.833392    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn" event={"ID":"7ebb2753-613c-41d7-b50c-dcc7e6705451","Type":"ContainerDied","Data":"39a0c3a5b4f5fa7d08b9acd0eaeffc93411ca3571291f86782eaecb3e4ca05a6"}
Sep 11 18:00:03.833872 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:03.833448    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39a0c3a5b4f5fa7d08b9acd0eaeffc93411ca3571291f86782eaecb3e4ca05a6"
Sep 11 18:00:04.061182 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.061127    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"
Sep 11 18:00:04.157662 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.157629    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"
Sep 11 18:00:04.162393 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.162330    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"
Sep 11 18:00:04.174367 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.171736    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"
Sep 11 18:00:04.844974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.844931    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2" event={"ID":"e4cc779f-e56c-4316-a8ce-b74132019e8d","Type":"ContainerDied","Data":"d87704cccaadac379f899c8afa51ea6642e7de973c8aed977fd321e3611b8b9c"}
Sep 11 18:00:04.845478 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.844999    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d87704cccaadac379f899c8afa51ea6642e7de973c8aed977fd321e3611b8b9c"
Sep 11 18:00:04.845478 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.845086    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"
Sep 11 18:00:04.859551 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.859523    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"
Sep 11 18:00:04.859685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.859524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd" event={"ID":"c2c7574f-9a06-4425-9627-593fc531cb6d","Type":"ContainerDied","Data":"500b57e568b94fa7a2d20565c0b9fd1eb39ff915504ffe2c04cd262fb6edfd9d"}
Sep 11 18:00:04.859685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.859670    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="500b57e568b94fa7a2d20565c0b9fd1eb39ff915504ffe2c04cd262fb6edfd9d"
Sep 11 18:00:04.861278 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.861260    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt"
Sep 11 18:00:04.861434 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.861312    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819160-6gstt" event={"ID":"44da4928-5a19-40bf-874d-df9d180b51de","Type":"ContainerDied","Data":"8328f733d01b1626745c840ab502a76d7582538a3c9bc8d6e13f19395f4d49d9"}
Sep 11 18:00:04.861434 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.861374    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8328f733d01b1626745c840ab502a76d7582538a3c9bc8d6e13f19395f4d49d9"
Sep 11 18:00:04.863012 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.862991    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"
Sep 11 18:00:04.863127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.863035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n" event={"ID":"77b9e412-d3ff-4783-a2da-6ab0eb0a9196","Type":"ContainerDied","Data":"e1993403de9adf0d79080a71c569627ca2d4d4d22a45e947fc5ea833fd25549f"}
Sep 11 18:00:04.863127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:04.863085    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1993403de9adf0d79080a71c569627ca2d4d4d22a45e947fc5ea833fd25549f"
Sep 11 18:00:06.904863 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:06.904828    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:00:08.701993 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:08.701955    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 18:00:08.702441 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:08.702008    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 18:00:08.714379 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:08.714335    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 18:00:08.898453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:08.898423    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 18:00:20.989163 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:20.989120    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:33.580465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580424    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"]
Sep 11 18:00:33.580918 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580899    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ebb2753-613c-41d7-b50c-dcc7e6705451" containerName="cleanup"
Sep 11 18:00:33.580918 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580910    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ebb2753-613c-41d7-b50c-dcc7e6705451" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580921    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77b9e412-d3ff-4783-a2da-6ab0eb0a9196" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580926    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="77b9e412-d3ff-4783-a2da-6ab0eb0a9196" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580937    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44da4928-5a19-40bf-874d-df9d180b51de" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580943    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="44da4928-5a19-40bf-874d-df9d180b51de" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580955    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4cc779f-e56c-4316-a8ce-b74132019e8d" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580960    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4cc779f-e56c-4316-a8ce-b74132019e8d" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580968    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2c7574f-9a06-4425-9627-593fc531cb6d" containerName="cleanup"
Sep 11 18:00:33.580994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.580973    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2c7574f-9a06-4425-9627-593fc531cb6d" containerName="cleanup"
Sep 11 18:00:33.581238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.581045    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4cc779f-e56c-4316-a8ce-b74132019e8d" containerName="cleanup"
Sep 11 18:00:33.581238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.581053    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2c7574f-9a06-4425-9627-593fc531cb6d" containerName="cleanup"
Sep 11 18:00:33.581238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.581060    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="44da4928-5a19-40bf-874d-df9d180b51de" containerName="cleanup"
Sep 11 18:00:33.581238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.581065    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="77b9e412-d3ff-4783-a2da-6ab0eb0a9196" containerName="cleanup"
Sep 11 18:00:33.581238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.581070    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ebb2753-613c-41d7-b50c-dcc7e6705451" containerName="cleanup"
Sep 11 18:00:33.585426 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.585409    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:33.588145 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.588124    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:00:33.605764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.605738    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"]
Sep 11 18:00:33.717470 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.717431    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f23d1cd-609c-4182-b248-6b7ff37d9495-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ed18394d-m88mw\" (UID: \"1f23d1cd-609c-4182-b248-6b7ff37d9495\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:33.759088 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.759057    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:33.818491 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.818456    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f23d1cd-609c-4182-b248-6b7ff37d9495-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ed18394d-m88mw\" (UID: \"1f23d1cd-609c-4182-b248-6b7ff37d9495\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:33.819085 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.819065    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f23d1cd-609c-4182-b248-6b7ff37d9495-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ed18394d-m88mw\" (UID: \"1f23d1cd-609c-4182-b248-6b7ff37d9495\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:33.895403 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:33.895329    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:34.186956 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:34.186932    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"]
Sep 11 18:00:34.188427 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:00:34.188404    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f23d1cd_609c_4182_b248_6b7ff37d9495.slice/crio-e639795e9eafc33afa3c567f65c1a59f40fc89c813789cfb4e58111fc1ee8e32 WatchSource:0}: Error finding container e639795e9eafc33afa3c567f65c1a59f40fc89c813789cfb4e58111fc1ee8e32: Status 404 returned error can't find the container with id e639795e9eafc33afa3c567f65c1a59f40fc89c813789cfb4e58111fc1ee8e32
Sep 11 18:00:35.044117 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:35.044024    2582 generic.go:356] "Generic (PLEG): container finished" podID="1f23d1cd-609c-4182-b248-6b7ff37d9495" containerID="c5810857964602e5d2b2b12e74d7caaba247aff3347a01d09db4d98ee529c7ff" exitCode=0
Sep 11 18:00:35.044117 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:35.044086    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw" event={"ID":"1f23d1cd-609c-4182-b248-6b7ff37d9495","Type":"ContainerDied","Data":"c5810857964602e5d2b2b12e74d7caaba247aff3347a01d09db4d98ee529c7ff"}
Sep 11 18:00:35.044522 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:35.044148    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw" event={"ID":"1f23d1cd-609c-4182-b248-6b7ff37d9495","Type":"ContainerStarted","Data":"e639795e9eafc33afa3c567f65c1a59f40fc89c813789cfb4e58111fc1ee8e32"}
Sep 11 18:00:35.285966 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:35.285932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:35.608483 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:35.608447    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:36.189809 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:36.189787    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:36.343435 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:36.343321    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f23d1cd-609c-4182-b248-6b7ff37d9495-maas-api-service-ca\") pod \"1f23d1cd-609c-4182-b248-6b7ff37d9495\" (UID: \"1f23d1cd-609c-4182-b248-6b7ff37d9495\") "
Sep 11 18:00:36.343669 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:36.343645    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f23d1cd-609c-4182-b248-6b7ff37d9495-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f23d1cd-609c-4182-b248-6b7ff37d9495" (UID: "1f23d1cd-609c-4182-b248-6b7ff37d9495"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:00:36.444333 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:36.444294    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f23d1cd-609c-4182-b248-6b7ff37d9495-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:00:36.472784 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:36.472752    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:37.057574 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.057543    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"
Sep 11 18:00:37.057731 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.057549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw" event={"ID":"1f23d1cd-609c-4182-b248-6b7ff37d9495","Type":"ContainerDied","Data":"e639795e9eafc33afa3c567f65c1a59f40fc89c813789cfb4e58111fc1ee8e32"}
Sep 11 18:00:37.057731 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.057671    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e639795e9eafc33afa3c567f65c1a59f40fc89c813789cfb4e58111fc1ee8e32"
Sep 11 18:00:37.173502 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.173468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:37.816075 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.816043    2582 scope.go:117] "RemoveContainer" containerID="e08785876ac59ca7e4fb7bd21a7772ff3e6592455ead60cec7b139fcd9001081"
Sep 11 18:00:37.825323 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.825302    2582 scope.go:117] "RemoveContainer" containerID="fd12515582791f10b6bf3d8f3e34e4c441d1e18dc688a31fea434b8e34b9288f"
Sep 11 18:00:37.831113 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:37.831063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:38.586241 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:38.586206    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:39.260694 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:39.260660    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:39.891337 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:39.891301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:44.803329 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:44.803291    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:00:48.374240 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:48.374203    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:49.349909 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.349871    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"]
Sep 11 18:00:49.350695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.350667    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:49.353144 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.353116    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-mt-ed18394d-29819160-bj8gd"]
Sep 11 18:00:49.442825 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.442792    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:49.518081 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.518049    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:49.602579 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.602471    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"]
Sep 11 18:00:49.604609 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.604583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:49.605790 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.605770    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ed18394d-m88mw"]
Sep 11 18:00:49.750313 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.750270    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f23d1cd-609c-4182-b248-6b7ff37d9495" path="/var/lib/kubelet/pods/1f23d1cd-609c-4182-b248-6b7ff37d9495/volumes"
Sep 11 18:00:49.750824 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:49.750802    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2c7574f-9a06-4425-9627-593fc531cb6d" path="/var/lib/kubelet/pods/c2c7574f-9a06-4425-9627-593fc531cb6d/volumes"
Sep 11 18:00:50.071367 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:50.071304    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:51.052297 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:51.052263    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:51.878342 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:51.878309    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:52.745428 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:52.745397    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:52.962757 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:52.962725    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:53.737473 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:53.737430    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:54.760235 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:54.760184    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:55.779199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:55.779165    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:57.932341 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:57.932306    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"]
Sep 11 18:00:57.982320 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:57.982287    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:00:59.420996 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:00:59.420960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:12.951849 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:12.951741    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq" podUID="1aadb12d-10d1-4533-baf3-c04f55f2219a" containerName="main" containerID="cri-o://90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c" gracePeriod=45
Sep 11 18:01:13.209509 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.209450    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 18:01:13.285063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.285023    2582 generic.go:356] "Generic (PLEG): container finished" podID="1aadb12d-10d1-4533-baf3-c04f55f2219a" containerID="90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c" exitCode=0
Sep 11 18:01:13.285228 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.285122    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"
Sep 11 18:01:13.285228 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.285159    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq" event={"ID":"1aadb12d-10d1-4533-baf3-c04f55f2219a","Type":"ContainerDied","Data":"90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c"}
Sep 11 18:01:13.285228 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.285204    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq" event={"ID":"1aadb12d-10d1-4533-baf3-c04f55f2219a","Type":"ContainerDied","Data":"e7acc7f78151d9dbc33623db1b0066f3c0079f7fcf2576283e7031f431b04646"}
Sep 11 18:01:13.285228 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.285221    2582 scope.go:117] "RemoveContainer" containerID="90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c"
Sep 11 18:01:13.294550 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.294529    2582 scope.go:117] "RemoveContainer" containerID="90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c"
Sep 11 18:01:13.294818 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:01:13.294798    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c\": container with ID starting with 90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c not found: ID does not exist" containerID="90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c"
Sep 11 18:01:13.294862 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.294827    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c"} err="failed to get container status \"90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c\": rpc error: code = NotFound desc = could not find container \"90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c\": container with ID starting with 90e70b732be3dec0766ab6e99ee10fe15a82341540d7c6d6bc662aa1441b6b2c not found: ID does not exist"
Sep 11 18:01:13.320189 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320163    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-tmp-dir\") pod \"1aadb12d-10d1-4533-baf3-c04f55f2219a\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") "
Sep 11 18:01:13.320295 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320225    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-model-cache\") pod \"1aadb12d-10d1-4533-baf3-c04f55f2219a\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") "
Sep 11 18:01:13.320295 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320250    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7rw9\" (UniqueName: \"kubernetes.io/projected/1aadb12d-10d1-4533-baf3-c04f55f2219a-kube-api-access-z7rw9\") pod \"1aadb12d-10d1-4533-baf3-c04f55f2219a\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") "
Sep 11 18:01:13.320414 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320306    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-dshm\") pod \"1aadb12d-10d1-4533-baf3-c04f55f2219a\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") "
Sep 11 18:01:13.320414 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320340    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-home\") pod \"1aadb12d-10d1-4533-baf3-c04f55f2219a\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") "
Sep 11 18:01:13.320517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320466    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1aadb12d-10d1-4533-baf3-c04f55f2219a" (UID: "1aadb12d-10d1-4533-baf3-c04f55f2219a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:13.320517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320486    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aadb12d-10d1-4533-baf3-c04f55f2219a-tls-certs\") pod \"1aadb12d-10d1-4533-baf3-c04f55f2219a\" (UID: \"1aadb12d-10d1-4533-baf3-c04f55f2219a\") "
Sep 11 18:01:13.320599 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320561    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-model-cache" (OuterVolumeSpecName: "model-cache") pod "1aadb12d-10d1-4533-baf3-c04f55f2219a" (UID: "1aadb12d-10d1-4533-baf3-c04f55f2219a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:13.320643 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320610    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-home" (OuterVolumeSpecName: "home") pod "1aadb12d-10d1-4533-baf3-c04f55f2219a" (UID: "1aadb12d-10d1-4533-baf3-c04f55f2219a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:13.320856 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320835    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:13.320992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320858    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:13.320992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.320872    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:13.322573 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.322551    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1aadb12d-10d1-4533-baf3-c04f55f2219a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1aadb12d-10d1-4533-baf3-c04f55f2219a" (UID: "1aadb12d-10d1-4533-baf3-c04f55f2219a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:01:13.322573 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.322560    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-dshm" (OuterVolumeSpecName: "dshm") pod "1aadb12d-10d1-4533-baf3-c04f55f2219a" (UID: "1aadb12d-10d1-4533-baf3-c04f55f2219a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:13.322699 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.322584    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1aadb12d-10d1-4533-baf3-c04f55f2219a-kube-api-access-z7rw9" (OuterVolumeSpecName: "kube-api-access-z7rw9") pod "1aadb12d-10d1-4533-baf3-c04f55f2219a" (UID: "1aadb12d-10d1-4533-baf3-c04f55f2219a"). InnerVolumeSpecName "kube-api-access-z7rw9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:01:13.421494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.421445    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7rw9\" (UniqueName: \"kubernetes.io/projected/1aadb12d-10d1-4533-baf3-c04f55f2219a-kube-api-access-z7rw9\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:13.421494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.421488    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1aadb12d-10d1-4533-baf3-c04f55f2219a-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:13.421494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.421497    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1aadb12d-10d1-4533-baf3-c04f55f2219a-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:13.606362 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.606315    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"]
Sep 11 18:01:13.608785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.608761    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ed18394d/e2e-lifecycle-model-ed18394d-kserve-5d599b55c9-rfjjq"]
Sep 11 18:01:13.748677 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:13.748641    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1aadb12d-10d1-4533-baf3-c04f55f2219a" path="/var/lib/kubelet/pods/1aadb12d-10d1-4533-baf3-c04f55f2219a/volumes"
Sep 11 18:01:17.350022 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:17.349988    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:18.436940 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:18.436902    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:18.705132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:18.705045    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:01:29.884400 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:29.884363    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:34.254284 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:34.254246    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:35.053647 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:35.053615    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:36.016232 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:36.016200    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:41.453971 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:41.453927    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:41.908332 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:41.908305    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:46.212688 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:46.212647    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:46.340414 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:46.340374    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:46.531947 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:46.531914    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"]
Sep 11 18:01:46.532199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:46.532173    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy" containerID="cri-o://45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9" gracePeriod=30
Sep 11 18:01:46.598779 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:46.598749    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:47.908880 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:47.908846    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:48.000066 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:48.000037    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:48.715334 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:48.715301    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:48.811253 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:48.811222    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:48.911576 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:48.911542    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:50.018877 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:50.018841    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:01:50.531496 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:50.531460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:50.632325 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:50.632293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:51.798205 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.798148    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 18:01:51.906254 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906217    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-certs\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906468 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906267    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-credential-socket\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906468 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906302    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-envoy\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906468 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906331    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b07338-2128-4d26-a72f-364a10c86a01-istiod-ca-cert\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906468 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906406    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-istio-token\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906468 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906450    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m28l9\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-kube-api-access-m28l9\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906742 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906495    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-socket\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906742 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906527    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:51.906742 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906578    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b07338-2128-4d26-a72f-364a10c86a01-istio-podinfo\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906742 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906624    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-data\") pod \"c7b07338-2128-4d26-a72f-364a10c86a01\" (UID: \"c7b07338-2128-4d26-a72f-364a10c86a01\") "
Sep 11 18:01:51.906742 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906649    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:51.906977 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906805    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:51.906977 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.906831    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7b07338-2128-4d26-a72f-364a10c86a01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:01:51.907101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.907031    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-data" (OuterVolumeSpecName: "istio-data") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:51.907180 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.907154    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:51.907232 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.907183    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:51.907232 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.907197    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:51.907232 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.907210    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7b07338-2128-4d26-a72f-364a10c86a01-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:51.907232 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.907224    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:51.908723 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.908702    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:01:51.908849 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.908724    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-istio-token" (OuterVolumeSpecName: "istio-token") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:01:51.909046 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.909018    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-kube-api-access-m28l9" (OuterVolumeSpecName: "kube-api-access-m28l9") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "kube-api-access-m28l9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:01:51.909154 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:51.909089    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7b07338-2128-4d26-a72f-364a10c86a01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c7b07338-2128-4d26-a72f-364a10c86a01" (UID: "c7b07338-2128-4d26-a72f-364a10c86a01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:01:52.008051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.008020    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7b07338-2128-4d26-a72f-364a10c86a01-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:52.008051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.008047    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7b07338-2128-4d26-a72f-364a10c86a01-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:52.008051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.008056    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:52.008268 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.008064    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m28l9\" (UniqueName: \"kubernetes.io/projected/c7b07338-2128-4d26-a72f-364a10c86a01-kube-api-access-m28l9\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:01:52.532002 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.531969    2582 generic.go:356] "Generic (PLEG): container finished" podID="c7b07338-2128-4d26-a72f-364a10c86a01" containerID="45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9" exitCode=0
Sep 11 18:01:52.532183 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.532015    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"
Sep 11 18:01:52.532183 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.532061    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" event={"ID":"c7b07338-2128-4d26-a72f-364a10c86a01","Type":"ContainerDied","Data":"45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9"}
Sep 11 18:01:52.532183 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.532120    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg" event={"ID":"c7b07338-2128-4d26-a72f-364a10c86a01","Type":"ContainerDied","Data":"d8ffd32bbeaf8cbcc6c42dc74ec113b13f7c9f41b0d1320b0fb38407bd54352d"}
Sep 11 18:01:52.532183 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.532142    2582 scope.go:117] "RemoveContainer" containerID="45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9"
Sep 11 18:01:52.542578 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.542558    2582 scope.go:117] "RemoveContainer" containerID="f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"
Sep 11 18:01:52.551749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.551729    2582 scope.go:117] "RemoveContainer" containerID="45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9"
Sep 11 18:01:52.552048 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:01:52.552020    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9\": container with ID starting with 45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9 not found: ID does not exist" containerID="45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9"
Sep 11 18:01:52.552137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.552057    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9"} err="failed to get container status \"45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9\": rpc error: code = NotFound desc = could not find container \"45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9\": container with ID starting with 45e6c15389c3a28fcec27b0f27727111ef2b6d63522629e16f69ba86eb4f7cd9 not found: ID does not exist"
Sep 11 18:01:52.552137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.552080    2582 scope.go:117] "RemoveContainer" containerID="f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"
Sep 11 18:01:52.552394 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:01:52.552372    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89\": container with ID starting with f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89 not found: ID does not exist" containerID="f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"
Sep 11 18:01:52.552477 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.552404    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89"} err="failed to get container status \"f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89\": rpc error: code = NotFound desc = could not find container \"f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89\": container with ID starting with f3b6542fbf69e2b24cb6f88fd0b86e26874ae81ea055b0b4d0d6ace6800ffc89 not found: ID does not exist"
Sep 11 18:01:52.553764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.553744    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"]
Sep 11 18:01:52.558165 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:52.558143    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ed18394d-openshift-default-577dfcb48-52bsg"]
Sep 11 18:01:53.747486 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:53.747450    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" path="/var/lib/kubelet/pods/c7b07338-2128-4d26-a72f-364a10c86a01/volumes"
Sep 11 18:01:56.025766 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:56.025724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:56.104772 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:56.104737    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:01:57.391692 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:01:57.391649    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:03.023526 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:03.023486    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:03.135456 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:03.135420    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:03.229538 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:03.229501    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:13.762423 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:13.762386    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:13.985135 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:13.985102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:18.810835 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:18.810802    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:24.939192 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:24.939158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:02:34.942142 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:34.942056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:37.865932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:37.865899    2582 scope.go:117] "RemoveContainer" containerID="13bb4d24b8a5aefe9ec4fd37ebf60340c4ff8a492060b5a895c98b5befd4b219"
Sep 11 18:02:37.874828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:37.874810    2582 scope.go:117] "RemoveContainer" containerID="ed1f524cc9737e5c8c7ab5855594d9ccd79e2c5752caad82c6a9efb248463c2f"
Sep 11 18:02:38.928892 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:38.928851    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:40.890251 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:40.890223    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:40.990533 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:40.990495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:41.275316 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.275276    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:41.547598 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.547524    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"]
Sep 11 18:02:41.548017 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548004    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1aadb12d-10d1-4533-baf3-c04f55f2219a" containerName="main"
Sep 11 18:02:41.548061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548019    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1aadb12d-10d1-4533-baf3-c04f55f2219a" containerName="main"
Sep 11 18:02:41.548061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548037    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy"
Sep 11 18:02:41.548061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548042    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy"
Sep 11 18:02:41.548061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548050    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy"
Sep 11 18:02:41.548061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548056    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy"
Sep 11 18:02:41.548061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548061    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f23d1cd-609c-4182-b248-6b7ff37d9495" containerName="revoke-keys"
Sep 11 18:02:41.548252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548069    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f23d1cd-609c-4182-b248-6b7ff37d9495" containerName="revoke-keys"
Sep 11 18:02:41.548252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548146    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f23d1cd-609c-4182-b248-6b7ff37d9495" containerName="revoke-keys"
Sep 11 18:02:41.548252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548158    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1aadb12d-10d1-4533-baf3-c04f55f2219a" containerName="main"
Sep 11 18:02:41.548252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548164    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy"
Sep 11 18:02:41.548252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.548172    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7b07338-2128-4d26-a72f-364a10c86a01" containerName="istio-proxy"
Sep 11 18:02:41.552775 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.552753    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.555297 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.555278    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-95f9c9-openshift-default-dockercfg-czkz6\""
Sep 11 18:02:41.564402 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.564378    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"]
Sep 11 18:02:41.587364 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c78c0172-5029-4441-98be-0a52a316a69a-istio-podinfo\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587379    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wf2xn\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-kube-api-access-wf2xn\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587411    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:41.587515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587463    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-certs\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587488    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-data\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-socket\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587577    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-envoy\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587622    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-istio-token\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587667    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-credential-socket\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.587824 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.587696    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c78c0172-5029-4441-98be-0a52a316a69a-istiod-ca-cert\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688481 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-credential-socket\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c78c0172-5029-4441-98be-0a52a316a69a-istiod-ca-cert\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688558    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c78c0172-5029-4441-98be-0a52a316a69a-istio-podinfo\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688601    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wf2xn\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-kube-api-access-wf2xn\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688829 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688668    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-certs\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688829 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-data\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688829 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688727    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-socket\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688829 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-envoy\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.688829 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.688817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-istio-token\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.689325 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.689217    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-certs\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.689325 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.689268    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c78c0172-5029-4441-98be-0a52a316a69a-istiod-ca-cert\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.689564 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.689383    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-socket\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.689654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.689630    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-data\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.690964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.690940    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-credential-socket\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.691422 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.691399    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-envoy\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.691561 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.691542    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c78c0172-5029-4441-98be-0a52a316a69a-istio-podinfo\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.697140 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.697111    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-istio-token\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.697463 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.697439    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wf2xn\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-kube-api-access-wf2xn\") pod \"e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") " pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:41.865514 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:41.865429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:42.064547 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.064508    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:42.142563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.142536    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"]
Sep 11 18:02:42.144954 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:02:42.144921    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc78c0172_5029_4441_98be_0a52a316a69a.slice/crio-853007531fd2b7fa7d282156e8d9970dd1d7a4151e698838679e213497e086d0 WatchSource:0}: Error finding container 853007531fd2b7fa7d282156e8d9970dd1d7a4151e698838679e213497e086d0: Status 404 returned error can't find the container with id 853007531fd2b7fa7d282156e8d9970dd1d7a4151e698838679e213497e086d0
Sep 11 18:02:42.147337 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.147305    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:02:42.147436 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.147390    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:02:42.147436 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.147420    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:02:42.827162 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.827125    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" event={"ID":"c78c0172-5029-4441-98be-0a52a316a69a","Type":"ContainerStarted","Data":"489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"}
Sep 11 18:02:42.827399 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.827188    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" event={"ID":"c78c0172-5029-4441-98be-0a52a316a69a","Type":"ContainerStarted","Data":"853007531fd2b7fa7d282156e8d9970dd1d7a4151e698838679e213497e086d0"}
Sep 11 18:02:42.847120 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.847069    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podStartSLOduration=1.847052681 podStartE2EDuration="1.847052681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:02:42.845989765 +0000 UTC m=+1445.716032709" watchObservedRunningTime="2026-09-11 18:02:42.847052681 +0000 UTC m=+1445.717095625"
Sep 11 18:02:42.866190 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:42.866154    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:43.871338 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:43.871302    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:44.148226 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:44.148119    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:44.810298 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:44.810264    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:44.838413 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:44.838381    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:44.839375 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:44.839326    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:02:44.907890 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:44.907849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:45.667861 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:45.667829    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:46.516486 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:46.516448    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:47.244560 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:47.244525    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:48.156759 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:48.156717    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:48.965726 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:48.965691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:49.772526 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:49.772495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:49.836748 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:49.836716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:50.073613 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:50.073527    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:50.226357 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:50.226313    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:50.339726 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:50.339632    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:55.074465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:55.074424    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:02:56.987221 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:02:56.987179    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:01.099809 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:01.099773    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:03:09.692388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.692334    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:09.813885 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.813849    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"]
Sep 11 18:03:09.818453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.818429    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:09.820744 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.820722    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:03:09.825517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.825484    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"]
Sep 11 18:03:09.884195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.884152    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce730ee-8e87-407c-b555-a851b9b991c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w\" (UID: \"5ce730ee-8e87-407c-b555-a851b9b991c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:09.985277 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.985239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce730ee-8e87-407c-b555-a851b9b991c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w\" (UID: \"5ce730ee-8e87-407c-b555-a851b9b991c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:09.986132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:09.986111    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce730ee-8e87-407c-b555-a851b9b991c3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w\" (UID: \"5ce730ee-8e87-407c-b555-a851b9b991c3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:10.048523 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:10.048481    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:10.133517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:10.133475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:10.409118 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:10.409084    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"]
Sep 11 18:03:10.410248 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:03:10.410219    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ce730ee_8e87_407c_b555_a851b9b991c3.slice/crio-8fbea6694c1ab131a3748dccaeb931037b5b82bf5d8cd7c4a6bbbf65ab170ba8 WatchSource:0}: Error finding container 8fbea6694c1ab131a3748dccaeb931037b5b82bf5d8cd7c4a6bbbf65ab170ba8: Status 404 returned error can't find the container with id 8fbea6694c1ab131a3748dccaeb931037b5b82bf5d8cd7c4a6bbbf65ab170ba8
Sep 11 18:03:10.772493 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:10.772468    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:11.006602 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:11.006508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w" event={"ID":"5ce730ee-8e87-407c-b555-a851b9b991c3","Type":"ContainerStarted","Data":"62c8743c8287537f81c131292ad9278898a737420b2d127c217068af900607e1"}
Sep 11 18:03:11.006602 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:11.006575    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w" event={"ID":"5ce730ee-8e87-407c-b555-a851b9b991c3","Type":"ContainerStarted","Data":"8fbea6694c1ab131a3748dccaeb931037b5b82bf5d8cd7c4a6bbbf65ab170ba8"}
Sep 11 18:03:11.023004 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:11.022955    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w" podStartSLOduration=2.022938868 podStartE2EDuration="2.022938868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:03:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:03:11.021111133 +0000 UTC m=+1473.891154079" watchObservedRunningTime="2026-09-11 18:03:11.022938868 +0000 UTC m=+1473.892981813"
Sep 11 18:03:11.594605 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:11.594574    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:12.014268 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.014230    2582 generic.go:356] "Generic (PLEG): container finished" podID="5ce730ee-8e87-407c-b555-a851b9b991c3" containerID="62c8743c8287537f81c131292ad9278898a737420b2d127c217068af900607e1" exitCode=0
Sep 11 18:03:12.014662 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.014268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w" event={"ID":"5ce730ee-8e87-407c-b555-a851b9b991c3","Type":"ContainerDied","Data":"62c8743c8287537f81c131292ad9278898a737420b2d127c217068af900607e1"}
Sep 11 18:03:12.015975 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.015952    2582 generic.go:356] "Generic (PLEG): container finished" podID="c78c0172-5029-4441-98be-0a52a316a69a" containerID="489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db" exitCode=0
Sep 11 18:03:12.016095 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.016023    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" event={"ID":"c78c0172-5029-4441-98be-0a52a316a69a","Type":"ContainerDied","Data":"489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"}
Sep 11 18:03:12.016435 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.016419    2582 scope.go:117] "RemoveContainer" containerID="489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"
Sep 11 18:03:12.017758 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.017729    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:03:12.017812 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.017798    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:03:12.017852 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.017831    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:03:12.274389 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:12.274278    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:13.024359 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.024322    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" event={"ID":"c78c0172-5029-4441-98be-0a52a316a69a","Type":"ContainerStarted","Data":"078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002"}
Sep 11 18:03:13.072077 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.072039    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:13.217988 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.217966    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:13.318060 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.317967    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce730ee-8e87-407c-b555-a851b9b991c3-maas-api-service-ca\") pod \"5ce730ee-8e87-407c-b555-a851b9b991c3\" (UID: \"5ce730ee-8e87-407c-b555-a851b9b991c3\") "
Sep 11 18:03:13.318322 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.318298    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ce730ee-8e87-407c-b555-a851b9b991c3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5ce730ee-8e87-407c-b555-a851b9b991c3" (UID: "5ce730ee-8e87-407c-b555-a851b9b991c3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:03:13.419167 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.419125    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5ce730ee-8e87-407c-b555-a851b9b991c3-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:13.868386 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.866571    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:03:13.868386 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.867721    2582 patch_prober.go:28] interesting pod/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 11 18:03:13.868386 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.867773    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 11 18:03:13.869137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:13.869106    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:14.031369 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.031280    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"
Sep 11 18:03:14.031369 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.031310    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w" event={"ID":"5ce730ee-8e87-407c-b555-a851b9b991c3","Type":"ContainerDied","Data":"8fbea6694c1ab131a3748dccaeb931037b5b82bf5d8cd7c4a6bbbf65ab170ba8"}
Sep 11 18:03:14.031776 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.031370    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8fbea6694c1ab131a3748dccaeb931037b5b82bf5d8cd7c4a6bbbf65ab170ba8"
Sep 11 18:03:14.595629 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.595589    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:14.704019 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.703982    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"]
Sep 11 18:03:14.826221 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.826186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:14.838484 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.838453    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:03:14.865992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.865891    2582 patch_prober.go:28] interesting pod/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 11 18:03:14.865992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:14.865977    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 11 18:03:15.373140 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:15.373098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:15.866643 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:15.866608    2582 patch_prober.go:28] interesting pod/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 11 18:03:15.866831 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:15.866663    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 11 18:03:16.458473 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:16.458419    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:16.866578 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:16.866533    2582 patch_prober.go:28] interesting pod/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 11 18:03:16.866752 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:16.866608    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 11 18:03:17.586952 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:17.586915    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:17.866556 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:17.866472    2582 patch_prober.go:28] interesting pod/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body=
Sep 11 18:03:17.866556 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:17.866544    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused"
Sep 11 18:03:18.315726 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:18.315691    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:18.869764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:18.869729    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:03:18.870890 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:18.870872    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:03:19.195801 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:19.195704    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:19.762442 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:19.762406    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:20.101667 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.101582    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:20.536223 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.536182    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:20.662796 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.662761    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:20.831918 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.831840    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"]
Sep 11 18:03:20.832564 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.832545    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ce730ee-8e87-407c-b555-a851b9b991c3" containerName="revoke-keys"
Sep 11 18:03:20.832640 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.832566    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ce730ee-8e87-407c-b555-a851b9b991c3" containerName="revoke-keys"
Sep 11 18:03:20.832700 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.832689    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ce730ee-8e87-407c-b555-a851b9b991c3" containerName="revoke-keys"
Sep 11 18:03:20.836553 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.836532    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.838706 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.838681    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a93550-openshift-default-dockercfg-dggzb\""
Sep 11 18:03:20.848129 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.848105    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"]
Sep 11 18:03:20.869999 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.869967    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:20.897224 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-istio-token\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897373 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897248    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5t6p\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-kube-api-access-f5t6p\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897373 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897279    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-data\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897475 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-certs\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897475 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897409    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-envoy\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897543 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897472    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-socket\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897543 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58553d9-aae7-44e2-823f-c413372d694c-istio-podinfo\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897651 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897541    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-credential-socket\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.897651 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.897586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58553d9-aae7-44e2-823f-c413372d694c-istiod-ca-cert\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998401 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998328    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-istio-token\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998406    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5t6p\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-kube-api-access-f5t6p\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-data\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998493    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-certs\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-envoy\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998567    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-socket\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998863 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998591    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58553d9-aae7-44e2-823f-c413372d694c-istio-podinfo\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998863 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998623    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-credential-socket\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.998863 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998660    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58553d9-aae7-44e2-823f-c413372d694c-istiod-ca-cert\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.999034 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-data\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.999034 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.998896    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-certs\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.999149 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.999088    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-socket\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.999319 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.999298    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-credential-socket\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:20.999600 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:20.999577    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58553d9-aae7-44e2-823f-c413372d694c-istiod-ca-cert\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:21.001317 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.001295    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-envoy\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:21.001422 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.001340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58553d9-aae7-44e2-823f-c413372d694c-istio-podinfo\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:21.006781 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.006750    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5t6p\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-kube-api-access-f5t6p\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:21.007021 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.007001    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-istio-token\") pod \"e2e-api-b-a93550-openshift-default-784758bccb-5598g\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") " pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:21.150935 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.150852    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:21.197853 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.197817    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:21.431738 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.431708    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"]
Sep 11 18:03:21.433065 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:03:21.433029    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf58553d9_aae7_44e2_823f_c413372d694c.slice/crio-9e3ab383d23c51eb4074fe79fd9e8c5205f2cb4a855b1668231928dddd841505 WatchSource:0}: Error finding container 9e3ab383d23c51eb4074fe79fd9e8c5205f2cb4a855b1668231928dddd841505: Status 404 returned error can't find the container with id 9e3ab383d23c51eb4074fe79fd9e8c5205f2cb4a855b1668231928dddd841505
Sep 11 18:03:21.435437 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.435402    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:03:21.435509 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.435491    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:03:21.435547 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:21.435521    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:03:22.094994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:22.094959    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" event={"ID":"f58553d9-aae7-44e2-823f-c413372d694c","Type":"ContainerStarted","Data":"5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"}
Sep 11 18:03:22.095201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:22.095033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" event={"ID":"f58553d9-aae7-44e2-823f-c413372d694c","Type":"ContainerStarted","Data":"9e3ab383d23c51eb4074fe79fd9e8c5205f2cb4a855b1668231928dddd841505"}
Sep 11 18:03:22.116275 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:22.116219    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podStartSLOduration=2.116202822 podStartE2EDuration="2.116202822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:03:22.113768684 +0000 UTC m=+1484.983811629" watchObservedRunningTime="2026-09-11 18:03:22.116202822 +0000 UTC m=+1484.986245767"
Sep 11 18:03:22.151445 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:22.151414    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:22.156943 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:22.156916    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:23.101162 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.101130    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:23.102229 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.102208    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:03:23.535543 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.535490    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"]
Sep 11 18:03:23.537859 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.537826    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-286c27-29819160-6p2g2"]
Sep 11 18:03:23.563714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.563683    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"]
Sep 11 18:03:23.563938 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.563912    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb" podUID="91898659-44fa-480c-a599-90fe5b0f2359" containerName="payload-processing" containerID="cri-o://c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42" gracePeriod=30
Sep 11 18:03:23.573445 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.573410    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"]
Sep 11 18:03:23.573655 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.573629    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm" podUID="d934c21b-959a-44bc-940b-7227316f73cc" containerName="payload-pre-processing" containerID="cri-o://650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f" gracePeriod=30
Sep 11 18:03:23.631971 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.631937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:23.724162 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.724111    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"]
Sep 11 18:03:23.727155 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.727126    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-286c27-n6k5w"]
Sep 11 18:03:23.753799 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.753495    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ce730ee-8e87-407c-b555-a851b9b991c3" path="/var/lib/kubelet/pods/5ce730ee-8e87-407c-b555-a851b9b991c3/volumes"
Sep 11 18:03:23.754132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.754094    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4cc779f-e56c-4316-a8ce-b74132019e8d" path="/var/lib/kubelet/pods/e4cc779f-e56c-4316-a8ce-b74132019e8d/volumes"
Sep 11 18:03:23.860566 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.860537    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 18:03:23.898582 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.898549    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 18:03:23.928341 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.928273    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d934c21b-959a-44bc-940b-7227316f73cc-plugins-config-volume\") pod \"d934c21b-959a-44bc-940b-7227316f73cc\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") "
Sep 11 18:03:23.928664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.928490    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22wd5\" (UniqueName: \"kubernetes.io/projected/d934c21b-959a-44bc-940b-7227316f73cc-kube-api-access-22wd5\") pod \"d934c21b-959a-44bc-940b-7227316f73cc\" (UID: \"d934c21b-959a-44bc-940b-7227316f73cc\") "
Sep 11 18:03:23.928664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.928529    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91898659-44fa-480c-a599-90fe5b0f2359-plugins-config-volume\") pod \"91898659-44fa-480c-a599-90fe5b0f2359\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") "
Sep 11 18:03:23.928664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.928575    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbbxj\" (UniqueName: \"kubernetes.io/projected/91898659-44fa-480c-a599-90fe5b0f2359-kube-api-access-rbbxj\") pod \"91898659-44fa-480c-a599-90fe5b0f2359\" (UID: \"91898659-44fa-480c-a599-90fe5b0f2359\") "
Sep 11 18:03:23.929791 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.929764    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/91898659-44fa-480c-a599-90fe5b0f2359-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "91898659-44fa-480c-a599-90fe5b0f2359" (UID: "91898659-44fa-480c-a599-90fe5b0f2359"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:03:23.930188 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.930165    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d934c21b-959a-44bc-940b-7227316f73cc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d934c21b-959a-44bc-940b-7227316f73cc" (UID: "d934c21b-959a-44bc-940b-7227316f73cc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:03:23.932023 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.932000    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d934c21b-959a-44bc-940b-7227316f73cc-kube-api-access-22wd5" (OuterVolumeSpecName: "kube-api-access-22wd5") pod "d934c21b-959a-44bc-940b-7227316f73cc" (UID: "d934c21b-959a-44bc-940b-7227316f73cc"). InnerVolumeSpecName "kube-api-access-22wd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:23.932376 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:23.932333    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/91898659-44fa-480c-a599-90fe5b0f2359-kube-api-access-rbbxj" (OuterVolumeSpecName: "kube-api-access-rbbxj") pod "91898659-44fa-480c-a599-90fe5b0f2359" (UID: "91898659-44fa-480c-a599-90fe5b0f2359"). InnerVolumeSpecName "kube-api-access-rbbxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:24.029571 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.029540    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22wd5\" (UniqueName: \"kubernetes.io/projected/d934c21b-959a-44bc-940b-7227316f73cc-kube-api-access-22wd5\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:24.029571 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.029566    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91898659-44fa-480c-a599-90fe5b0f2359-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:24.029571 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.029576    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbbxj\" (UniqueName: \"kubernetes.io/projected/91898659-44fa-480c-a599-90fe5b0f2359-kube-api-access-rbbxj\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:24.029797 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.029584    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d934c21b-959a-44bc-940b-7227316f73cc-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:24.108279 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.108196    2582 generic.go:356] "Generic (PLEG): container finished" podID="d934c21b-959a-44bc-940b-7227316f73cc" containerID="650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f" exitCode=0
Sep 11 18:03:24.108279 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.108240    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"
Sep 11 18:03:24.108530 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.108280    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm" event={"ID":"d934c21b-959a-44bc-940b-7227316f73cc","Type":"ContainerDied","Data":"650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f"}
Sep 11 18:03:24.108530 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.108331    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm" event={"ID":"d934c21b-959a-44bc-940b-7227316f73cc","Type":"ContainerDied","Data":"7a0499aafac8df47543db514077fcbde4d8979dae3a321dfdc7c9b1a6093c871"}
Sep 11 18:03:24.108530 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.108371    2582 scope.go:117] "RemoveContainer" containerID="650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f"
Sep 11 18:03:24.109853 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.109830    2582 generic.go:356] "Generic (PLEG): container finished" podID="91898659-44fa-480c-a599-90fe5b0f2359" containerID="c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42" exitCode=0
Sep 11 18:03:24.109950 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.109876    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"
Sep 11 18:03:24.109950 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.109920    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb" event={"ID":"91898659-44fa-480c-a599-90fe5b0f2359","Type":"ContainerDied","Data":"c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42"}
Sep 11 18:03:24.110065 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.109973    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb" event={"ID":"91898659-44fa-480c-a599-90fe5b0f2359","Type":"ContainerDied","Data":"885a04a1a56776a60c95df01dc7096ff4742d09f1e8a7dfbdaf933c661c67329"}
Sep 11 18:03:24.119625 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.119609    2582 scope.go:117] "RemoveContainer" containerID="650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f"
Sep 11 18:03:24.119900 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:03:24.119874    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f\": container with ID starting with 650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f not found: ID does not exist" containerID="650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f"
Sep 11 18:03:24.120035 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.119911    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f"} err="failed to get container status \"650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f\": rpc error: code = NotFound desc = could not find container \"650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f\": container with ID starting with 650a6ba982c19a2b3d34bab4c88407873a9752d9f6384aeca9a847846478854f not found: ID does not exist"
Sep 11 18:03:24.120035 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.119933    2582 scope.go:117] "RemoveContainer" containerID="c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42"
Sep 11 18:03:24.129275 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.129257    2582 scope.go:117] "RemoveContainer" containerID="c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42"
Sep 11 18:03:24.129582 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:03:24.129563    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42\": container with ID starting with c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42 not found: ID does not exist" containerID="c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42"
Sep 11 18:03:24.129652 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.129591    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42"} err="failed to get container status \"c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42\": rpc error: code = NotFound desc = could not find container \"c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42\": container with ID starting with c9f59f8561db99332639c5b77461c6d59721c615d2f1c5a806bb4caf8f19ed42 not found: ID does not exist"
Sep 11 18:03:24.138089 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.138065    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"]
Sep 11 18:03:24.146154 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.146123    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-286c27-bbb5cdbf8-nzqrm"]
Sep 11 18:03:24.161066 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.161042    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"]
Sep 11 18:03:24.167413 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.167385    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-286c27-7b454ddb6c-fs5nb"]
Sep 11 18:03:24.999091 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:24.999056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:25.098586 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:25.098549    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:25.583401 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:25.583365    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:25.756992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:25.756953    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="91898659-44fa-480c-a599-90fe5b0f2359" path="/var/lib/kubelet/pods/91898659-44fa-480c-a599-90fe5b0f2359/volumes"
Sep 11 18:03:25.757373 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:25.757331    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d934c21b-959a-44bc-940b-7227316f73cc" path="/var/lib/kubelet/pods/d934c21b-959a-44bc-940b-7227316f73cc/volumes"
Sep 11 18:03:26.263657 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:26.263621    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:26.915090 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:26.915059    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:27.621040 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:27.621009    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:28.370709 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:28.370677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:29.180597 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:29.180559    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:29.267126 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:29.267070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:29.574801 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:29.574764    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:29.679844 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:29.679783    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:29.723260 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:29.723223    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p" podUID="dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" containerName="main" containerID="cri-o://4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9" gracePeriod=45
Sep 11 18:03:29.827151 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:29.827063    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:30.003187 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.003164    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 18:03:30.090382 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.090256    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:30.093934 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.093912    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-home\") pod \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") "
Sep 11 18:03:30.094050 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.093982    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5lgh\" (UniqueName: \"kubernetes.io/projected/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-kube-api-access-s5lgh\") pod \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") "
Sep 11 18:03:30.094050 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094024    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-model-cache\") pod \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") "
Sep 11 18:03:30.094147 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094092    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-dshm\") pod \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") "
Sep 11 18:03:30.094147 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094130    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tls-certs\") pod \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") "
Sep 11 18:03:30.094253 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094158    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-home" (OuterVolumeSpecName: "home") pod "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" (UID: "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.094253 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094193    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tmp-dir\") pod \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\" (UID: \"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade\") "
Sep 11 18:03:30.094437 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094385    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-model-cache" (OuterVolumeSpecName: "model-cache") pod "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" (UID: "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.094540 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094520    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" (UID: "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.094675 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094654    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.094759 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094677    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.094759 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.094690    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.096504 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.096476    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" (UID: "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:03:30.096649 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.096632    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-kube-api-access-s5lgh" (OuterVolumeSpecName: "kube-api-access-s5lgh") pod "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" (UID: "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade"). InnerVolumeSpecName "kube-api-access-s5lgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:03:30.096703 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.096632    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-dshm" (OuterVolumeSpecName: "dshm") pod "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" (UID: "dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:03:30.152959 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.152923    2582 generic.go:356] "Generic (PLEG): container finished" podID="dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" containerID="4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9" exitCode=0
Sep 11 18:03:30.153139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.152974    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"
Sep 11 18:03:30.153139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.152984    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p" event={"ID":"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade","Type":"ContainerDied","Data":"4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9"}
Sep 11 18:03:30.153139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.153022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p" event={"ID":"dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade","Type":"ContainerDied","Data":"e0a56259fea8f3a9d0832742fabadd212e652d2103249ae196aea767d976a0a4"}
Sep 11 18:03:30.153139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.153044    2582 scope.go:117] "RemoveContainer" containerID="4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9"
Sep 11 18:03:30.166516 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.166490    2582 scope.go:117] "RemoveContainer" containerID="4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9"
Sep 11 18:03:30.166863 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:03:30.166838    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9\": container with ID starting with 4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9 not found: ID does not exist" containerID="4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9"
Sep 11 18:03:30.166967 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.166870    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9"} err="failed to get container status \"4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9\": rpc error: code = NotFound desc = could not find container \"4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9\": container with ID starting with 4cf207ca9edc8472cf944c7b7a13defc07475a776fd01b35159eba9da8176ee9 not found: ID does not exist"
Sep 11 18:03:30.176661 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.176633    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"]
Sep 11 18:03:30.181857 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.181830    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-286c27/distinct-2-w0-286c27-kserve-7fccf6bbf4-8pb8p"]
Sep 11 18:03:30.195777 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.195754    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.195777 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.195776    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.195932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.195786    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5lgh\" (UniqueName: \"kubernetes.io/projected/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade-kube-api-access-s5lgh\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:03:30.774343 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:30.774307    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:31.198759 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:31.198677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:31.273875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:31.273842    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:31.462877 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:31.462781    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:31.749138 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:31.749103    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" path="/var/lib/kubelet/pods/dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade/volumes"
Sep 11 18:03:37.696435 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:37.696409    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:03:37.701084 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:37.701058    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:03:50.986283 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:50.986242    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:51.086050 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:51.086013    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:51.188464 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:51.188423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:51.933567 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:51.933533    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:52.504913 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:52.504880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:03:52.690030 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:52.689989    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:52.921432 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:52.921334    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:53.477731 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:53.477694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:54.235954 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:54.235920    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:54.343280 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:54.343242    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:55.029671 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:55.029641    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:55.784448 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:55.784407    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:56.466517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:56.466470    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:57.188617 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.188583    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:57.321303 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321268    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"]
Sep 11 18:03:57.321780 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321759    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" containerName="main"
Sep 11 18:03:57.321780 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321778    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" containerName="main"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321798    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="91898659-44fa-480c-a599-90fe5b0f2359" containerName="payload-processing"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321804    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="91898659-44fa-480c-a599-90fe5b0f2359" containerName="payload-processing"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321822    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d934c21b-959a-44bc-940b-7227316f73cc" containerName="payload-pre-processing"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321828    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d934c21b-959a-44bc-940b-7227316f73cc" containerName="payload-pre-processing"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321896    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="91898659-44fa-480c-a599-90fe5b0f2359" containerName="payload-processing"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321904    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbb319f5-b5b2-4cdb-81bc-9332bbcf1ade" containerName="main"
Sep 11 18:03:57.321926 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.321912    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d934c21b-959a-44bc-940b-7227316f73cc" containerName="payload-pre-processing"
Sep 11 18:03:57.325115 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.325096    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:03:57.327731 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.327596    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:03:57.330504 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.330480    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"]
Sep 11 18:03:57.356967 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.356925    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1609c026-9f95-41ff-aaf6-114d621b3216-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x\" (UID: \"1609c026-9f95-41ff-aaf6-114d621b3216\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:03:57.459030 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.458926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1609c026-9f95-41ff-aaf6-114d621b3216-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x\" (UID: \"1609c026-9f95-41ff-aaf6-114d621b3216\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:03:57.460179 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.460012    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1609c026-9f95-41ff-aaf6-114d621b3216-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x\" (UID: \"1609c026-9f95-41ff-aaf6-114d621b3216\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:03:57.648432 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.648391    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:03:57.671018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.670983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:57.936647 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:57.936607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"]
Sep 11 18:03:58.332058 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:58.332022    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x" event={"ID":"1609c026-9f95-41ff-aaf6-114d621b3216","Type":"ContainerStarted","Data":"9415d67206280bc1a6a41e0578e910a31153b0bc59dd9078b439f34225360133"}
Sep 11 18:03:59.068093 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:59.067995    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:59.276404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:59.276364    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:03:59.345950 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:59.345374    2582 generic.go:356] "Generic (PLEG): container finished" podID="1609c026-9f95-41ff-aaf6-114d621b3216" containerID="6e013a9ce598d3494e7b9cdebb4482f043d70f5a001c6845df291e7150d786c5" exitCode=0
Sep 11 18:03:59.345950 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:59.345573    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x" event={"ID":"1609c026-9f95-41ff-aaf6-114d621b3216","Type":"ContainerDied","Data":"6e013a9ce598d3494e7b9cdebb4482f043d70f5a001c6845df291e7150d786c5"}
Sep 11 18:03:59.519063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:03:59.519032    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:00.062457 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:00.062405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:00.508653 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:00.508624    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:04:00.601688 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:00.601648    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1609c026-9f95-41ff-aaf6-114d621b3216-maas-api-service-ca\") pod \"1609c026-9f95-41ff-aaf6-114d621b3216\" (UID: \"1609c026-9f95-41ff-aaf6-114d621b3216\") "
Sep 11 18:04:00.602025 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:00.602000    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1609c026-9f95-41ff-aaf6-114d621b3216-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1609c026-9f95-41ff-aaf6-114d621b3216" (UID: "1609c026-9f95-41ff-aaf6-114d621b3216"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:00.702853 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:00.702771    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1609c026-9f95-41ff-aaf6-114d621b3216-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:00.820162 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:00.820121    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:01.360761 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:01.360732    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"
Sep 11 18:04:01.360761 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:01.360742    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x" event={"ID":"1609c026-9f95-41ff-aaf6-114d621b3216","Type":"ContainerDied","Data":"9415d67206280bc1a6a41e0578e910a31153b0bc59dd9078b439f34225360133"}
Sep 11 18:04:01.360961 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:01.360782    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9415d67206280bc1a6a41e0578e910a31153b0bc59dd9078b439f34225360133"
Sep 11 18:04:01.543770 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:01.543729    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:02.371950 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:02.371903    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:03.071840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.071808    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:03.375203 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.375113    2582 generic.go:356] "Generic (PLEG): container finished" podID="f58553d9-aae7-44e2-823f-c413372d694c" containerID="5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc" exitCode=0
Sep 11 18:04:03.375203 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.375158    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" event={"ID":"f58553d9-aae7-44e2-823f-c413372d694c","Type":"ContainerDied","Data":"5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"}
Sep 11 18:04:03.375684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.375664    2582 scope.go:117] "RemoveContainer" containerID="5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"
Sep 11 18:04:03.376893 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.376862    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:04:03.376950 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.376934    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:04:03.377018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.376999    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:04:03.864212 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:03.864174    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:04.383542 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:04.383508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" event={"ID":"f58553d9-aae7-44e2-823f-c413372d694c","Type":"ContainerStarted","Data":"7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec"}
Sep 11 18:04:05.110229 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:05.110196    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:05.153828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:05.153380    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:04:05.153828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:05.153515    2582 patch_prober.go:28] interesting pod/e2e-api-b-a93550-openshift-default-784758bccb-5598g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 18:04:05.153828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:05.153568    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 18:04:06.151715 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:06.151673    2582 patch_prober.go:28] interesting pod/e2e-api-b-a93550-openshift-default-784758bccb-5598g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 18:04:06.152295 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:06.152262    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 18:04:07.151429 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:07.151391    2582 patch_prober.go:28] interesting pod/e2e-api-b-a93550-openshift-default-784758bccb-5598g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 18:04:07.151719 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:07.151686    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 18:04:08.101271 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:08.101237    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:04:08.151846 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:08.151803    2582 patch_prober.go:28] interesting pod/e2e-api-b-a93550-openshift-default-784758bccb-5598g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 18:04:08.152048 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:08.151874    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 18:04:09.151817 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:09.151772    2582 patch_prober.go:28] interesting pod/e2e-api-b-a93550-openshift-default-784758bccb-5598g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body=
Sep 11 18:04:09.152200 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:09.151845    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused"
Sep 11 18:04:10.156275 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:10.156245    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:04:10.157397 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:10.157374    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:04:12.805670 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:12.805629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:12.907684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:12.907628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:13.006264 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.006230    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:13.014031 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.013990    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"]
Sep 11 18:04:13.016504 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.016479    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-95f9c9-crx9x"]
Sep 11 18:04:13.112163 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.112070    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:13.425621 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.425531    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:13.525597 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.525555    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:13.747827 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:13.747798    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1609c026-9f95-41ff-aaf6-114d621b3216" path="/var/lib/kubelet/pods/1609c026-9f95-41ff-aaf6-114d621b3216/volumes"
Sep 11 18:04:14.266459 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:14.266426    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:15.010399 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:15.010333    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:15.713566 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:15.713531    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:16.690280 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:16.690248    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:17.374720 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:17.374688    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:18.251396 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:18.251340    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:19.664501 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:19.664455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:19.771096 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:19.771056    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:19.938381 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:19.938277    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"]
Sep 11 18:04:19.938606 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:19.938577    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy" containerID="cri-o://078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002" gracePeriod=30
Sep 11 18:04:19.968059 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:19.968024    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:20.298909 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.298874    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"]
Sep 11 18:04:20.299311 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.299299    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1609c026-9f95-41ff-aaf6-114d621b3216" containerName="revoke-keys"
Sep 11 18:04:20.299385 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.299313    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1609c026-9f95-41ff-aaf6-114d621b3216" containerName="revoke-keys"
Sep 11 18:04:20.299425 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.299415    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1609c026-9f95-41ff-aaf6-114d621b3216" containerName="revoke-keys"
Sep 11 18:04:20.302820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.302796    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:20.304783 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.304762    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:04:20.311488 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.311465    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"]
Sep 11 18:04:20.403557 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.403515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5090061a-bbc0-4ac5-aa0f-e824363aac87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a93550-dlqnv\" (UID: \"5090061a-bbc0-4ac5-aa0f-e824363aac87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:20.445959 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.445929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:20.504864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.504833    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5090061a-bbc0-4ac5-aa0f-e824363aac87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a93550-dlqnv\" (UID: \"5090061a-bbc0-4ac5-aa0f-e824363aac87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:20.505571 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.505542    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5090061a-bbc0-4ac5-aa0f-e824363aac87-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a93550-dlqnv\" (UID: \"5090061a-bbc0-4ac5-aa0f-e824363aac87\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:20.615117 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.615027    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:20.888332 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:20.888302    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"]
Sep 11 18:04:20.889962 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:04:20.889925    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5090061a_bbc0_4ac5_aa0f_e824363aac87.slice/crio-d5ad5b459a6fa8d5e524bf3c20132fc02bd0cc8b1a32760903b16a0e4bc2b1d7 WatchSource:0}: Error finding container d5ad5b459a6fa8d5e524bf3c20132fc02bd0cc8b1a32760903b16a0e4bc2b1d7: Status 404 returned error can't find the container with id d5ad5b459a6fa8d5e524bf3c20132fc02bd0cc8b1a32760903b16a0e4bc2b1d7
Sep 11 18:04:21.263169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:21.263134    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:21.495752 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:21.495715    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv" event={"ID":"5090061a-bbc0-4ac5-aa0f-e824363aac87","Type":"ContainerStarted","Data":"9e0c3f0a50513d9008be85c2dfef796b27f6045ec4b09a85adf92eb890bfd49f"}
Sep 11 18:04:21.495953 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:21.495771    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv" event={"ID":"5090061a-bbc0-4ac5-aa0f-e824363aac87","Type":"ContainerStarted","Data":"d5ad5b459a6fa8d5e524bf3c20132fc02bd0cc8b1a32760903b16a0e4bc2b1d7"}
Sep 11 18:04:21.510626 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:21.510580    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv" podStartSLOduration=1.510551588 podStartE2EDuration="1.510551588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:04:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:04:21.50918235 +0000 UTC m=+1544.379225294" watchObservedRunningTime="2026-09-11 18:04:21.510551588 +0000 UTC m=+1544.380594545"
Sep 11 18:04:22.057391 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:22.057328    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:22.503465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:22.503432    2582 generic.go:356] "Generic (PLEG): container finished" podID="5090061a-bbc0-4ac5-aa0f-e824363aac87" containerID="9e0c3f0a50513d9008be85c2dfef796b27f6045ec4b09a85adf92eb890bfd49f" exitCode=0
Sep 11 18:04:22.503465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:22.503467    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv" event={"ID":"5090061a-bbc0-4ac5-aa0f-e824363aac87","Type":"ContainerDied","Data":"9e0c3f0a50513d9008be85c2dfef796b27f6045ec4b09a85adf92eb890bfd49f"}
Sep 11 18:04:22.666071 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:22.666040    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:22.770851 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:22.770766    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:23.389511 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:23.389474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:23.651664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:23.651604    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:23.740692 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:23.740666    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5090061a-bbc0-4ac5-aa0f-e824363aac87-maas-api-service-ca\") pod \"5090061a-bbc0-4ac5-aa0f-e824363aac87\" (UID: \"5090061a-bbc0-4ac5-aa0f-e824363aac87\") "
Sep 11 18:04:23.741069 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:23.741047    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5090061a-bbc0-4ac5-aa0f-e824363aac87-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5090061a-bbc0-4ac5-aa0f-e824363aac87" (UID: "5090061a-bbc0-4ac5-aa0f-e824363aac87"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:23.741155 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:23.741139    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5090061a-bbc0-4ac5-aa0f-e824363aac87-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:24.289289 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:24.289255    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:24.517980 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:24.517943    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"
Sep 11 18:04:24.518497 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:24.517979    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv" event={"ID":"5090061a-bbc0-4ac5-aa0f-e824363aac87","Type":"ContainerDied","Data":"d5ad5b459a6fa8d5e524bf3c20132fc02bd0cc8b1a32760903b16a0e4bc2b1d7"}
Sep 11 18:04:24.518497 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:24.518065    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5ad5b459a6fa8d5e524bf3c20132fc02bd0cc8b1a32760903b16a0e4bc2b1d7"
Sep 11 18:04:24.987099 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:24.987072    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:25.199946 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.199890    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:04:25.357701 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357667    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wf2xn\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-kube-api-access-wf2xn\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.357894 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357719    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c78c0172-5029-4441-98be-0a52a316a69a-istio-podinfo\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.357894 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357761    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-certs\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.357894 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-istio-token\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.357894 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357814    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-data\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.357894 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-credential-socket\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.357894 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357866    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-envoy\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357918    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-socket\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.357986    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c78c0172-5029-4441-98be-0a52a316a69a-istiod-ca-cert\") pod \"c78c0172-5029-4441-98be-0a52a316a69a\" (UID: \"c78c0172-5029-4441-98be-0a52a316a69a\") "
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-data" (OuterVolumeSpecName: "istio-data") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358207    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358218    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358464    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358492    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358464    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358505    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.358684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.358547    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c78c0172-5029-4441-98be-0a52a316a69a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:25.360143 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.360116    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-istio-token" (OuterVolumeSpecName: "istio-token") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:25.360235 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.360151    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:25.360341 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.360324    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-kube-api-access-wf2xn" (OuterVolumeSpecName: "kube-api-access-wf2xn") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "kube-api-access-wf2xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:25.360518 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.360495    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c78c0172-5029-4441-98be-0a52a316a69a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c78c0172-5029-4441-98be-0a52a316a69a" (UID: "c78c0172-5029-4441-98be-0a52a316a69a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:04:25.459176 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.459106    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c78c0172-5029-4441-98be-0a52a316a69a-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.459176 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.459134    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.459176 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.459143    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.459176 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.459151    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c78c0172-5029-4441-98be-0a52a316a69a-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.459176 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.459160    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c78c0172-5029-4441-98be-0a52a316a69a-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.459176 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.459169    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wf2xn\" (UniqueName: \"kubernetes.io/projected/c78c0172-5029-4441-98be-0a52a316a69a-kube-api-access-wf2xn\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:25.525995 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.525965    2582 generic.go:356] "Generic (PLEG): container finished" podID="c78c0172-5029-4441-98be-0a52a316a69a" containerID="078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002" exitCode=0
Sep 11 18:04:25.526336 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.526016    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"
Sep 11 18:04:25.526336 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.526053    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" event={"ID":"c78c0172-5029-4441-98be-0a52a316a69a","Type":"ContainerDied","Data":"078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002"}
Sep 11 18:04:25.526336 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.526110    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl" event={"ID":"c78c0172-5029-4441-98be-0a52a316a69a","Type":"ContainerDied","Data":"853007531fd2b7fa7d282156e8d9970dd1d7a4151e698838679e213497e086d0"}
Sep 11 18:04:25.526336 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.526140    2582 scope.go:117] "RemoveContainer" containerID="078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002"
Sep 11 18:04:25.536602 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.536587    2582 scope.go:117] "RemoveContainer" containerID="489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"
Sep 11 18:04:25.545197 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.545180    2582 scope.go:117] "RemoveContainer" containerID="078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002"
Sep 11 18:04:25.545517 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:25.545497    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002\": container with ID starting with 078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002 not found: ID does not exist" containerID="078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002"
Sep 11 18:04:25.545595 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.545526    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002"} err="failed to get container status \"078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002\": rpc error: code = NotFound desc = could not find container \"078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002\": container with ID starting with 078ed0362491133cc6f5eebf5457902b9c428bd3938c6ce1b342efdaae168002 not found: ID does not exist"
Sep 11 18:04:25.545595 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.545546    2582 scope.go:117] "RemoveContainer" containerID="489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"
Sep 11 18:04:25.545843 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:25.545825    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db\": container with ID starting with 489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db not found: ID does not exist" containerID="489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"
Sep 11 18:04:25.545902 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.545852    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db"} err="failed to get container status \"489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db\": rpc error: code = NotFound desc = could not find container \"489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db\": container with ID starting with 489de41716ba2e8664c0c345b86f6f285f7c217e05bd7fc2c9ba63c8c8fcb2db not found: ID does not exist"
Sep 11 18:04:25.551154 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.551133    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"]
Sep 11 18:04:25.557033 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.557011    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-95f9c9-openshift-default-857f578d4b-pvwjl"]
Sep 11 18:04:25.749363 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.749319    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c78c0172-5029-4441-98be-0a52a316a69a" path="/var/lib/kubelet/pods/c78c0172-5029-4441-98be-0a52a316a69a/volumes"
Sep 11 18:04:25.798661 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:25.798630    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:26.612402 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:26.612369    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:04:29.639734 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:29.639694    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:30.597652 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:30.597620    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:33.586015 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.585978    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:33.709525 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709486    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"]
Sep 11 18:04:33.709960 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709946    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy"
Sep 11 18:04:33.710029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709962    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy"
Sep 11 18:04:33.710029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709972    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5090061a-bbc0-4ac5-aa0f-e824363aac87" containerName="revoke-keys"
Sep 11 18:04:33.710029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709978    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5090061a-bbc0-4ac5-aa0f-e824363aac87" containerName="revoke-keys"
Sep 11 18:04:33.710029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709992    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy"
Sep 11 18:04:33.710029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.709998    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy"
Sep 11 18:04:33.710211 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.710078    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy"
Sep 11 18:04:33.710211 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.710087    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5090061a-bbc0-4ac5-aa0f-e824363aac87" containerName="revoke-keys"
Sep 11 18:04:33.710211 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.710095    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c78c0172-5029-4441-98be-0a52a316a69a" containerName="istio-proxy"
Sep 11 18:04:33.714211 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.714191    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:33.716514 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.716487    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:04:33.719414 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.719390    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"]
Sep 11 18:04:33.749834 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.749799    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/369eb84e-a564-43d3-819d-8631b543ecfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh\" (UID: \"369eb84e-a564-43d3-819d-8631b543ecfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:33.851227 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.851145    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/369eb84e-a564-43d3-819d-8631b543ecfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh\" (UID: \"369eb84e-a564-43d3-819d-8631b543ecfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:33.851843 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:33.851818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/369eb84e-a564-43d3-819d-8631b543ecfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh\" (UID: \"369eb84e-a564-43d3-819d-8631b543ecfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:34.026420 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.026389    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:34.044896 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.044865    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:34.298480 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.298453    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"]
Sep 11 18:04:34.300945 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:04:34.300916    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369eb84e_a564_43d3_819d_8631b543ecfa.slice/crio-6eb711153d4326f85932d06afc06848449d42420e701d0599632a67e4cfdd556 WatchSource:0}: Error finding container 6eb711153d4326f85932d06afc06848449d42420e701d0599632a67e4cfdd556: Status 404 returned error can't find the container with id 6eb711153d4326f85932d06afc06848449d42420e701d0599632a67e4cfdd556
Sep 11 18:04:34.586371 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.586333    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh" event={"ID":"369eb84e-a564-43d3-819d-8631b543ecfa","Type":"ContainerStarted","Data":"7063dac1ee54aa8446c4dab0ab4659bc1014250a37e75584deb6b704bbba312d"}
Sep 11 18:04:34.586730 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.586399    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh" event={"ID":"369eb84e-a564-43d3-819d-8631b543ecfa","Type":"ContainerStarted","Data":"6eb711153d4326f85932d06afc06848449d42420e701d0599632a67e4cfdd556"}
Sep 11 18:04:34.601312 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.601266    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh" podStartSLOduration=1.601251098 podStartE2EDuration="1.601251098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:04:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:04:34.600050968 +0000 UTC m=+1557.470093915" watchObservedRunningTime="2026-09-11 18:04:34.601251098 +0000 UTC m=+1557.471294079"
Sep 11 18:04:34.900768 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:34.900668    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:35.572806 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:35.572773    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:35.592908 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:35.592868    2582 generic.go:356] "Generic (PLEG): container finished" podID="369eb84e-a564-43d3-819d-8631b543ecfa" containerID="7063dac1ee54aa8446c4dab0ab4659bc1014250a37e75584deb6b704bbba312d" exitCode=0
Sep 11 18:04:35.593277 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:35.592952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh" event={"ID":"369eb84e-a564-43d3-819d-8631b543ecfa","Type":"ContainerDied","Data":"7063dac1ee54aa8446c4dab0ab4659bc1014250a37e75584deb6b704bbba312d"}
Sep 11 18:04:36.248690 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.248654    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:36.329977 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.329937    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:36.429305 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.429273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:36.492627 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.492596    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:36.632519 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.632463    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"]
Sep 11 18:04:36.636778 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.636743    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a93550-dlqnv"]
Sep 11 18:04:36.646937 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.646876    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:36.722234 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.722204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:36.761525 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.761501    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:36.782112 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.782083    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/369eb84e-a564-43d3-819d-8631b543ecfa-maas-api-service-ca\") pod \"369eb84e-a564-43d3-819d-8631b543ecfa\" (UID: \"369eb84e-a564-43d3-819d-8631b543ecfa\") "
Sep 11 18:04:36.782672 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.782492    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/369eb84e-a564-43d3-819d-8631b543ecfa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "369eb84e-a564-43d3-819d-8631b543ecfa" (UID: "369eb84e-a564-43d3-819d-8631b543ecfa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:36.883819 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:36.883741    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/369eb84e-a564-43d3-819d-8631b543ecfa-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:37.044304 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.044268    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:37.441633 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.441584    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:37.611954 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.611917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh" event={"ID":"369eb84e-a564-43d3-819d-8631b543ecfa","Type":"ContainerDied","Data":"6eb711153d4326f85932d06afc06848449d42420e701d0599632a67e4cfdd556"}
Sep 11 18:04:37.611954 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.611949    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"
Sep 11 18:04:37.612159 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.611997    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6eb711153d4326f85932d06afc06848449d42420e701d0599632a67e4cfdd556"
Sep 11 18:04:37.747634 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.747601    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5090061a-bbc0-4ac5-aa0f-e824363aac87" path="/var/lib/kubelet/pods/5090061a-bbc0-4ac5-aa0f-e824363aac87/volumes"
Sep 11 18:04:37.925449 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.925419    2582 scope.go:117] "RemoveContainer" containerID="28f7612c1ca0f89e6bc618f337467eba5b8bfda270c40bfbcca15d1d1ed2f718"
Sep 11 18:04:37.934404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.934384    2582 scope.go:117] "RemoveContainer" containerID="66306600fb2ca5eaba1baf46c7e852ef2d73386f6969a6c6be1191bad111e34a"
Sep 11 18:04:37.944285 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:37.944265    2582 scope.go:117] "RemoveContainer" containerID="270e48e6d04c0d80955dd2c2fdce534e379ddfb9ca543d24d5749a2fe8a76dfb"
Sep 11 18:04:38.148093 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:38.148008    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:38.604090 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:38.604055    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"]
Sep 11 18:04:38.617523 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:38.617493    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"]
Sep 11 18:04:38.682639 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:38.682603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:39.445037 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:39.445000    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:39.799752 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:39.799707    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:40.456123 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:40.456088    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:41.241149 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:41.241118    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:42.920703 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:42.920670    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:42.993640 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:42.993573    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:43.166447 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.166416    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"]
Sep 11 18:04:43.166663 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.166630    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy" containerID="cri-o://7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec" gracePeriod=30
Sep 11 18:04:43.180064 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.179983    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:43.544608 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.544569    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"]
Sep 11 18:04:43.545029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.545015    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="369eb84e-a564-43d3-819d-8631b543ecfa" containerName="revoke-keys"
Sep 11 18:04:43.545075 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.545030    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="369eb84e-a564-43d3-819d-8631b543ecfa" containerName="revoke-keys"
Sep 11 18:04:43.545116 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.545107    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="369eb84e-a564-43d3-819d-8631b543ecfa" containerName="revoke-keys"
Sep 11 18:04:43.548491 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.548469    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:43.550609 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.550585    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:04:43.555214 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.555189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"]
Sep 11 18:04:43.655404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.655369    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65e3544a-881c-4359-9245-96bb1bce89a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55\" (UID: \"65e3544a-881c-4359-9245-96bb1bce89a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:43.678007 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.677973    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:43.756343 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.756311    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65e3544a-881c-4359-9245-96bb1bce89a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55\" (UID: \"65e3544a-881c-4359-9245-96bb1bce89a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:43.756942 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.756923    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65e3544a-881c-4359-9245-96bb1bce89a2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55\" (UID: \"65e3544a-881c-4359-9245-96bb1bce89a2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:43.862382 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:43.862283    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:44.132668 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:44.132638    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"]
Sep 11 18:04:44.133868 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:04:44.133840    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65e3544a_881c_4359_9245_96bb1bce89a2.slice/crio-036c1c9f90e052c53184f97165fe8220145bfa32ba8fbf8dbac60e4307d4cbc0 WatchSource:0}: Error finding container 036c1c9f90e052c53184f97165fe8220145bfa32ba8fbf8dbac60e4307d4cbc0: Status 404 returned error can't find the container with id 036c1c9f90e052c53184f97165fe8220145bfa32ba8fbf8dbac60e4307d4cbc0
Sep 11 18:04:44.473889 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:44.473791    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:44.657043 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:44.657006    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55" event={"ID":"65e3544a-881c-4359-9245-96bb1bce89a2","Type":"ContainerStarted","Data":"8a2114dd438b798d13a0f4b8f1fae38ba34e1cf738734c8557430adb78d4dae1"}
Sep 11 18:04:44.657207 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:44.657055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55" event={"ID":"65e3544a-881c-4359-9245-96bb1bce89a2","Type":"ContainerStarted","Data":"036c1c9f90e052c53184f97165fe8220145bfa32ba8fbf8dbac60e4307d4cbc0"}
Sep 11 18:04:44.674820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:44.674761    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55" podStartSLOduration=1.674746709 podStartE2EDuration="1.674746709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:04:44.67335909 +0000 UTC m=+1567.543402027" watchObservedRunningTime="2026-09-11 18:04:44.674746709 +0000 UTC m=+1567.544789654"
Sep 11 18:04:45.464455 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:45.464412    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:45.665474 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:45.665442    2582 generic.go:356] "Generic (PLEG): container finished" podID="65e3544a-881c-4359-9245-96bb1bce89a2" containerID="8a2114dd438b798d13a0f4b8f1fae38ba34e1cf738734c8557430adb78d4dae1" exitCode=0
Sep 11 18:04:45.665632 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:45.665532    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55" event={"ID":"65e3544a-881c-4359-9245-96bb1bce89a2","Type":"ContainerDied","Data":"8a2114dd438b798d13a0f4b8f1fae38ba34e1cf738734c8557430adb78d4dae1"}
Sep 11 18:04:46.189631 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:46.189593    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:46.812864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:46.812839    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:46.888293 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:46.888264    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65e3544a-881c-4359-9245-96bb1bce89a2-maas-api-service-ca\") pod \"65e3544a-881c-4359-9245-96bb1bce89a2\" (UID: \"65e3544a-881c-4359-9245-96bb1bce89a2\") "
Sep 11 18:04:46.888667 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:46.888640    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65e3544a-881c-4359-9245-96bb1bce89a2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "65e3544a-881c-4359-9245-96bb1bce89a2" (UID: "65e3544a-881c-4359-9245-96bb1bce89a2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:46.957438 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:46.957337    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:46.989170 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:46.989136    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/65e3544a-881c-4359-9245-96bb1bce89a2-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:47.625187 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:47.625131    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:47.677669 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:47.677637    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"
Sep 11 18:04:47.677820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:47.677670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55" event={"ID":"65e3544a-881c-4359-9245-96bb1bce89a2","Type":"ContainerDied","Data":"036c1c9f90e052c53184f97165fe8220145bfa32ba8fbf8dbac60e4307d4cbc0"}
Sep 11 18:04:47.677820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:47.677720    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="036c1c9f90e052c53184f97165fe8220145bfa32ba8fbf8dbac60e4307d4cbc0"
Sep 11 18:04:48.414105 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.414081    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:04:48.504390 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504357    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58553d9-aae7-44e2-823f-c413372d694c-istio-podinfo\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504423    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-istio-token\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504518    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-envoy\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504547    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-credential-socket\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504691 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504598    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5t6p\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-kube-api-access-f5t6p\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504691 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504622    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-certs\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504691 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504645    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58553d9-aae7-44e2-823f-c413372d694c-istiod-ca-cert\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504836 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504695    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-socket\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504836 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504740    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-data\") pod \"f58553d9-aae7-44e2-823f-c413372d694c\" (UID: \"f58553d9-aae7-44e2-823f-c413372d694c\") "
Sep 11 18:04:48.504948 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504861    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:48.504948 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.504872    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:48.505110 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.505069    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-data" (OuterVolumeSpecName: "istio-data") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:48.505110 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.505079    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:48.505239 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.505116    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f58553d9-aae7-44e2-823f-c413372d694c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:48.505239 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.505151    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.505239 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.505170    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.506731 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.506703    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-istio-token" (OuterVolumeSpecName: "istio-token") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:48.506975 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.506945    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f58553d9-aae7-44e2-823f-c413372d694c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:04:48.506975 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.506962    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-kube-api-access-f5t6p" (OuterVolumeSpecName: "kube-api-access-f5t6p") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "kube-api-access-f5t6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:48.507084 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.506970    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f58553d9-aae7-44e2-823f-c413372d694c" (UID: "f58553d9-aae7-44e2-823f-c413372d694c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:48.606405 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606373    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.606405 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606400    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5t6p\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-kube-api-access-f5t6p\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.606405 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606410    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58553d9-aae7-44e2-823f-c413372d694c-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.606614 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606420    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.606614 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606428    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58553d9-aae7-44e2-823f-c413372d694c-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.606614 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606437    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58553d9-aae7-44e2-823f-c413372d694c-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.606614 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.606444    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58553d9-aae7-44e2-823f-c413372d694c-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:48.695362 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.695316    2582 generic.go:356] "Generic (PLEG): container finished" podID="f58553d9-aae7-44e2-823f-c413372d694c" containerID="7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec" exitCode=0
Sep 11 18:04:48.695526 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.695387    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"
Sep 11 18:04:48.695526 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.695391    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" event={"ID":"f58553d9-aae7-44e2-823f-c413372d694c","Type":"ContainerDied","Data":"7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec"}
Sep 11 18:04:48.695526 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.695525    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g" event={"ID":"f58553d9-aae7-44e2-823f-c413372d694c","Type":"ContainerDied","Data":"9e3ab383d23c51eb4074fe79fd9e8c5205f2cb4a855b1668231928dddd841505"}
Sep 11 18:04:48.695653 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.695546    2582 scope.go:117] "RemoveContainer" containerID="7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec"
Sep 11 18:04:48.706054 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.706036    2582 scope.go:117] "RemoveContainer" containerID="5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"
Sep 11 18:04:48.714439 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.714420    2582 scope.go:117] "RemoveContainer" containerID="7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec"
Sep 11 18:04:48.714685 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:48.714666    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec\": container with ID starting with 7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec not found: ID does not exist" containerID="7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec"
Sep 11 18:04:48.714747 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.714694    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec"} err="failed to get container status \"7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec\": rpc error: code = NotFound desc = could not find container \"7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec\": container with ID starting with 7e1a7789709a89928cfc9c78ae9a71815ef30a5b397dcdfd25dc77e3a99e6dec not found: ID does not exist"
Sep 11 18:04:48.714747 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.714713    2582 scope.go:117] "RemoveContainer" containerID="5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"
Sep 11 18:04:48.714951 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:48.714933    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc\": container with ID starting with 5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc not found: ID does not exist" containerID="5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"
Sep 11 18:04:48.714986 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.714957    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc"} err="failed to get container status \"5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc\": rpc error: code = NotFound desc = could not find container \"5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc\": container with ID starting with 5fc93b50f37c27b8580c73114687bf07ee851776a8c52a1df12838cdfc3f8ecc not found: ID does not exist"
Sep 11 18:04:48.720244 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.720219    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"]
Sep 11 18:04:48.726006 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:48.725985    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a93550-openshift-default-784758bccb-5598g"]
Sep 11 18:04:49.747230 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:49.747197    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f58553d9-aae7-44e2-823f-c413372d694c" path="/var/lib/kubelet/pods/f58553d9-aae7-44e2-823f-c413372d694c/volumes"
Sep 11 18:04:50.193037 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.192954    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:50.218598 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.218562    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"]
Sep 11 18:04:50.218804 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.218781    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp" podUID="3c0b81a9-78e6-48d2-b55a-119556f9a879" containerName="payload-processing" containerID="cri-o://73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344" gracePeriod=30
Sep 11 18:04:50.305314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.305282    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:50.389621 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.389589    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"]
Sep 11 18:04:50.393905 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.393880    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w4-4ab253-w6rmh"]
Sep 11 18:04:50.396319 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.396293    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:50.474318 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.474071    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:50.490939 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.490916    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 18:04:50.622534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.622501    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c0b81a9-78e6-48d2-b55a-119556f9a879-plugins-config-volume\") pod \"3c0b81a9-78e6-48d2-b55a-119556f9a879\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") "
Sep 11 18:04:50.622676 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.622567    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f6t86\" (UniqueName: \"kubernetes.io/projected/3c0b81a9-78e6-48d2-b55a-119556f9a879-kube-api-access-f6t86\") pod \"3c0b81a9-78e6-48d2-b55a-119556f9a879\" (UID: \"3c0b81a9-78e6-48d2-b55a-119556f9a879\") "
Sep 11 18:04:50.622980 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.622955    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c0b81a9-78e6-48d2-b55a-119556f9a879-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3c0b81a9-78e6-48d2-b55a-119556f9a879" (UID: "3c0b81a9-78e6-48d2-b55a-119556f9a879"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:04:50.624748 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.624726    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c0b81a9-78e6-48d2-b55a-119556f9a879-kube-api-access-f6t86" (OuterVolumeSpecName: "kube-api-access-f6t86") pod "3c0b81a9-78e6-48d2-b55a-119556f9a879" (UID: "3c0b81a9-78e6-48d2-b55a-119556f9a879"). InnerVolumeSpecName "kube-api-access-f6t86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:50.708856 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.708822    2582 generic.go:356] "Generic (PLEG): container finished" podID="3c0b81a9-78e6-48d2-b55a-119556f9a879" containerID="73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344" exitCode=0
Sep 11 18:04:50.709018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.708858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp" event={"ID":"3c0b81a9-78e6-48d2-b55a-119556f9a879","Type":"ContainerDied","Data":"73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344"}
Sep 11 18:04:50.709018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.708873    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"
Sep 11 18:04:50.709018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.708929    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp" event={"ID":"3c0b81a9-78e6-48d2-b55a-119556f9a879","Type":"ContainerDied","Data":"ca4bf34d0f9363a4e1c19a9f789e49d5b9edb2002d6315e604fe58f87108127f"}
Sep 11 18:04:50.709018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.708966    2582 scope.go:117] "RemoveContainer" containerID="73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344"
Sep 11 18:04:50.718522 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.718506    2582 scope.go:117] "RemoveContainer" containerID="73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344"
Sep 11 18:04:50.718807 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:50.718785    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344\": container with ID starting with 73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344 not found: ID does not exist" containerID="73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344"
Sep 11 18:04:50.718900 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.718811    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344"} err="failed to get container status \"73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344\": rpc error: code = NotFound desc = could not find container \"73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344\": container with ID starting with 73915b0284ac350879a82810c3095f5f443a47673858104d6bc7f319db172344 not found: ID does not exist"
Sep 11 18:04:50.723785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.723765    2582 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3c0b81a9-78e6-48d2-b55a-119556f9a879-plugins-config-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:50.723785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.723785    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f6t86\" (UniqueName: \"kubernetes.io/projected/3c0b81a9-78e6-48d2-b55a-119556f9a879-kube-api-access-f6t86\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:50.731118 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.731098    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"]
Sep 11 18:04:50.735962 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.735938    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w4-4ab253-7d5776b677-f8lpp"]
Sep 11 18:04:50.795417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:50.795389    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:51.351337 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:51.351303    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:51.747161 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:51.747130    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="369eb84e-a564-43d3-819d-8631b543ecfa" path="/var/lib/kubelet/pods/369eb84e-a564-43d3-819d-8631b543ecfa/volumes"
Sep 11 18:04:51.747537 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:51.747522    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c0b81a9-78e6-48d2-b55a-119556f9a879" path="/var/lib/kubelet/pods/3c0b81a9-78e6-48d2-b55a-119556f9a879/volumes"
Sep 11 18:04:52.190116 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:52.190023    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:53.017992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:53.017955    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:53.624397 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:53.624335    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z" podUID="e6e9993e-16ed-49e5-8b06-32aec1d81aa5" containerName="main" containerID="cri-o://22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687" gracePeriod=45
Sep 11 18:04:53.638274 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:53.638240    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs" podUID="292e1415-a3af-4e48-98a7-1b2d1d1e76b5" containerName="main" containerID="cri-o://0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd" gracePeriod=45
Sep 11 18:04:53.685630 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:53.685605    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:53.931511 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:53.931481    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 18:04:53.934638 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:53.934611    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 18:04:54.057686 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057652    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-home\") pod \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") "
Sep 11 18:04:54.057875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057693    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-model-cache\") pod \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") "
Sep 11 18:04:54.057875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057724    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-dshm\") pod \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") "
Sep 11 18:04:54.057875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057789    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tls-certs\") pod \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") "
Sep 11 18:04:54.057875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057875    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tmp-dir\") pod \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") "
Sep 11 18:04:54.058094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057921    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-model-cache\") pod \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") "
Sep 11 18:04:54.058094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057954    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5chwb\" (UniqueName: \"kubernetes.io/projected/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-kube-api-access-5chwb\") pod \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") "
Sep 11 18:04:54.058094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057959    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-home" (OuterVolumeSpecName: "home") pod "292e1415-a3af-4e48-98a7-1b2d1d1e76b5" (UID: "292e1415-a3af-4e48-98a7-1b2d1d1e76b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.058094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057973    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "292e1415-a3af-4e48-98a7-1b2d1d1e76b5" (UID: "292e1415-a3af-4e48-98a7-1b2d1d1e76b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.058094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.057986    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-dshm\") pod \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") "
Sep 11 18:04:54.058094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058062    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tls-certs\") pod \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") "
Sep 11 18:04:54.058407 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058107    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8b77t\" (UniqueName: \"kubernetes.io/projected/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-kube-api-access-8b77t\") pod \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\" (UID: \"292e1415-a3af-4e48-98a7-1b2d1d1e76b5\") "
Sep 11 18:04:54.058407 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058171    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-home\") pod \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") "
Sep 11 18:04:54.058407 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058180    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "292e1415-a3af-4e48-98a7-1b2d1d1e76b5" (UID: "292e1415-a3af-4e48-98a7-1b2d1d1e76b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.058407 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058200    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tmp-dir\") pod \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\" (UID: \"e6e9993e-16ed-49e5-8b06-32aec1d81aa5\") "
Sep 11 18:04:54.058598 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058522    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-model-cache" (OuterVolumeSpecName: "model-cache") pod "e6e9993e-16ed-49e5-8b06-32aec1d81aa5" (UID: "e6e9993e-16ed-49e5-8b06-32aec1d81aa5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.058661 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058634    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.058661 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058654    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.058764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058721    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.058764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058734    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.058764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058731    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e6e9993e-16ed-49e5-8b06-32aec1d81aa5" (UID: "e6e9993e-16ed-49e5-8b06-32aec1d81aa5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.058946 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.058926    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-home" (OuterVolumeSpecName: "home") pod "e6e9993e-16ed-49e5-8b06-32aec1d81aa5" (UID: "e6e9993e-16ed-49e5-8b06-32aec1d81aa5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.060463 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.060321    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e6e9993e-16ed-49e5-8b06-32aec1d81aa5" (UID: "e6e9993e-16ed-49e5-8b06-32aec1d81aa5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:04:54.060463 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.060413    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "292e1415-a3af-4e48-98a7-1b2d1d1e76b5" (UID: "292e1415-a3af-4e48-98a7-1b2d1d1e76b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:04:54.060463 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.060413    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-dshm" (OuterVolumeSpecName: "dshm") pod "e6e9993e-16ed-49e5-8b06-32aec1d81aa5" (UID: "e6e9993e-16ed-49e5-8b06-32aec1d81aa5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.060645 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.060488    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-dshm" (OuterVolumeSpecName: "dshm") pod "292e1415-a3af-4e48-98a7-1b2d1d1e76b5" (UID: "292e1415-a3af-4e48-98a7-1b2d1d1e76b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:04:54.060645 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.060516    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-kube-api-access-5chwb" (OuterVolumeSpecName: "kube-api-access-5chwb") pod "e6e9993e-16ed-49e5-8b06-32aec1d81aa5" (UID: "e6e9993e-16ed-49e5-8b06-32aec1d81aa5"). InnerVolumeSpecName "kube-api-access-5chwb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:54.060645 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.060614    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-kube-api-access-8b77t" (OuterVolumeSpecName: "kube-api-access-8b77t") pod "292e1415-a3af-4e48-98a7-1b2d1d1e76b5" (UID: "292e1415-a3af-4e48-98a7-1b2d1d1e76b5"). InnerVolumeSpecName "kube-api-access-8b77t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159437    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8b77t\" (UniqueName: \"kubernetes.io/projected/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-kube-api-access-8b77t\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159467    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159480    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159494    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159502    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159510    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5chwb\" (UniqueName: \"kubernetes.io/projected/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-kube-api-access-5chwb\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159521    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/292e1415-a3af-4e48-98a7-1b2d1d1e76b5-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.159534 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.159533    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e6e9993e-16ed-49e5-8b06-32aec1d81aa5-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:04:54.402381 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.402335    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:54.409326 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.409297    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"]
Sep 11 18:04:54.411823 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.411766    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-21e472-29819160-psb4n"]
Sep 11 18:04:54.503673 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.503644    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:54.566021 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.565986    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:54.602840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.602800    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"]
Sep 11 18:04:54.604809 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.604788    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-21e472-b8p55"]
Sep 11 18:04:54.627836 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.627806    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:54.735732 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.735704    2582 generic.go:356] "Generic (PLEG): container finished" podID="e6e9993e-16ed-49e5-8b06-32aec1d81aa5" containerID="22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687" exitCode=0
Sep 11 18:04:54.735911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.735749    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"
Sep 11 18:04:54.735911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.735788    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z" event={"ID":"e6e9993e-16ed-49e5-8b06-32aec1d81aa5","Type":"ContainerDied","Data":"22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687"}
Sep 11 18:04:54.735911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.735834    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z" event={"ID":"e6e9993e-16ed-49e5-8b06-32aec1d81aa5","Type":"ContainerDied","Data":"caea1e051ce5ac61ba7e8948effbee2ef70860abf346b36caa479bd56f653122"}
Sep 11 18:04:54.735911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.735857    2582 scope.go:117] "RemoveContainer" containerID="22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687"
Sep 11 18:04:54.737289 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.737157    2582 generic.go:356] "Generic (PLEG): container finished" podID="292e1415-a3af-4e48-98a7-1b2d1d1e76b5" containerID="0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd" exitCode=0
Sep 11 18:04:54.737289 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.737200    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"
Sep 11 18:04:54.737289 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.737247    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs" event={"ID":"292e1415-a3af-4e48-98a7-1b2d1d1e76b5","Type":"ContainerDied","Data":"0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd"}
Sep 11 18:04:54.737289 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.737287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs" event={"ID":"292e1415-a3af-4e48-98a7-1b2d1d1e76b5","Type":"ContainerDied","Data":"b6c3adf2102d1e41d9c19e292a9d850f41bdfb9a5761c1e21e36edebd40f18f1"}
Sep 11 18:04:54.747555 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.747534    2582 scope.go:117] "RemoveContainer" containerID="22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687"
Sep 11 18:04:54.748331 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:54.747941    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687\": container with ID starting with 22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687 not found: ID does not exist" containerID="22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687"
Sep 11 18:04:54.748331 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.747977    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687"} err="failed to get container status \"22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687\": rpc error: code = NotFound desc = could not find container \"22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687\": container with ID starting with 22855e8e16c9828423b5282f3039d5ecb294061486f51c3e3d4ede81a1597687 not found: ID does not exist"
Sep 11 18:04:54.748331 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.748013    2582 scope.go:117] "RemoveContainer" containerID="0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd"
Sep 11 18:04:54.757003 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.756983    2582 scope.go:117] "RemoveContainer" containerID="0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd"
Sep 11 18:04:54.757273 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:04:54.757250    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd\": container with ID starting with 0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd not found: ID does not exist" containerID="0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd"
Sep 11 18:04:54.757517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.757279    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd"} err="failed to get container status \"0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd\": rpc error: code = NotFound desc = could not find container \"0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd\": container with ID starting with 0a69d093c575bc6e5e28337ed1c9e2ca02deebc3d29134be1eee3c965886d1dd not found: ID does not exist"
Sep 11 18:04:54.765465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.765439    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"]
Sep 11 18:04:54.768362 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.768327    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/premium-w4-4ab253-kserve-94fbfc785-zkz7z"]
Sep 11 18:04:54.777452 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.777408    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"]
Sep 11 18:04:54.783313 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.783294    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-4ab253/simulator-w4-4ab253-kserve-bf8fbc9cb-dwtvs"]
Sep 11 18:04:54.995151 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:54.995043    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:55.096061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.096010    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:55.747535 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.747499    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="292e1415-a3af-4e48-98a7-1b2d1d1e76b5" path="/var/lib/kubelet/pods/292e1415-a3af-4e48-98a7-1b2d1d1e76b5/volumes"
Sep 11 18:04:55.748006 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.747871    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65e3544a-881c-4359-9245-96bb1bce89a2" path="/var/lib/kubelet/pods/65e3544a-881c-4359-9245-96bb1bce89a2/volumes"
Sep 11 18:04:55.748126 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.748111    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77b9e412-d3ff-4783-a2da-6ab0eb0a9196" path="/var/lib/kubelet/pods/77b9e412-d3ff-4783-a2da-6ab0eb0a9196/volumes"
Sep 11 18:04:55.748505 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.748489    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6e9993e-16ed-49e5-8b06-32aec1d81aa5" path="/var/lib/kubelet/pods/e6e9993e-16ed-49e5-8b06-32aec1d81aa5/volumes"
Sep 11 18:04:55.877270 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.877233    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:55.985927 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:55.985897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:56.746839 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:56.746805    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:57.488054 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:57.488017    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:57.998310 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:57.998275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:58.100965 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:58.100929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:58.282383 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:58.282290    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"]
Sep 11 18:04:58.282580 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:58.282557    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy" containerID="cri-o://c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395" gracePeriod=30
Sep 11 18:04:58.310393 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:58.310341    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:04:59.597105 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:04:59.597062    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:05:01.427883 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:01.427849    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:01.528230 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:01.528201    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:01.701613 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:01.701528    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:02.047252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047214    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"]
Sep 11 18:05:02.047641 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047628    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c0b81a9-78e6-48d2-b55a-119556f9a879" containerName="payload-processing"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047643    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c0b81a9-78e6-48d2-b55a-119556f9a879" containerName="payload-processing"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047658    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047663    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047674    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047680    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047693    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65e3544a-881c-4359-9245-96bb1bce89a2" containerName="revoke-keys"
Sep 11 18:05:02.047695 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047698    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="65e3544a-881c-4359-9245-96bb1bce89a2" containerName="revoke-keys"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047707    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6e9993e-16ed-49e5-8b06-32aec1d81aa5" containerName="main"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047712    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6e9993e-16ed-49e5-8b06-32aec1d81aa5" containerName="main"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047718    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292e1415-a3af-4e48-98a7-1b2d1d1e76b5" containerName="main"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047725    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="292e1415-a3af-4e48-98a7-1b2d1d1e76b5" containerName="main"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047782    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="292e1415-a3af-4e48-98a7-1b2d1d1e76b5" containerName="main"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047789    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6e9993e-16ed-49e5-8b06-32aec1d81aa5" containerName="main"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047795    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="65e3544a-881c-4359-9245-96bb1bce89a2" containerName="revoke-keys"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047802    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047808    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c0b81a9-78e6-48d2-b55a-119556f9a879" containerName="payload-processing"
Sep 11 18:05:02.047912 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.047816    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58553d9-aae7-44e2-823f-c413372d694c" containerName="istio-proxy"
Sep 11 18:05:02.052296 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.052276    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:02.054443 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.054422    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:05:02.058512 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.058488    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"]
Sep 11 18:05:02.209055 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.209021    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:02.239605 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.239574    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1578cc-9956-4d6c-bc17-ddce49c36cbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9\" (UID: \"ae1578cc-9956-4d6c-bc17-ddce49c36cbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:02.340207 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.340119    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1578cc-9956-4d6c-bc17-ddce49c36cbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9\" (UID: \"ae1578cc-9956-4d6c-bc17-ddce49c36cbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:02.340811 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.340791    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1578cc-9956-4d6c-bc17-ddce49c36cbd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9\" (UID: \"ae1578cc-9956-4d6c-bc17-ddce49c36cbd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:02.364044 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.364017    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:02.663125 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.663047    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"]
Sep 11 18:05:02.668148 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:05:02.668115    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1578cc_9956_4d6c_bc17_ddce49c36cbd.slice/crio-07de26d628e8889b719933739adf4bf890c12c6f2e017a7530badf322a5804d9 WatchSource:0}: Error finding container 07de26d628e8889b719933739adf4bf890c12c6f2e017a7530badf322a5804d9: Status 404 returned error can't find the container with id 07de26d628e8889b719933739adf4bf890c12c6f2e017a7530badf322a5804d9
Sep 11 18:05:02.785651 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:02.785618    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9" event={"ID":"ae1578cc-9956-4d6c-bc17-ddce49c36cbd","Type":"ContainerStarted","Data":"07de26d628e8889b719933739adf4bf890c12c6f2e017a7530badf322a5804d9"}
Sep 11 18:05:03.534262 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.534240    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 18:05:03.577669 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.577635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:03.652636 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652603    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-data\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.652636 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652636    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-credential-socket\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.652849 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652665    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-istio-token\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.652849 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652699    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb540766-20db-4486-8989-ae0e2b6de57c-istio-podinfo\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.652849 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652724    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-certs\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.653008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652849    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-socket\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.653008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652872    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-envoy\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.653008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652908    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb540766-20db-4486-8989-ae0e2b6de57c-istiod-ca-cert\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.653008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652931    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-kube-api-access-2hdh4\") pod \"cb540766-20db-4486-8989-ae0e2b6de57c\" (UID: \"cb540766-20db-4486-8989-ae0e2b6de57c\") "
Sep 11 18:05:03.653261 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652927    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:03.653410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.652940    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-data" (OuterVolumeSpecName: "istio-data") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:03.653410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.653275    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:03.653529 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.653427    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:03.653529 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.653452    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb540766-20db-4486-8989-ae0e2b6de57c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:05:03.655529 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.655507    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:03.655529 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.655516    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-kube-api-access-2hdh4" (OuterVolumeSpecName: "kube-api-access-2hdh4") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "kube-api-access-2hdh4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:03.655529 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.655512    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cb540766-20db-4486-8989-ae0e2b6de57c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:05:03.655701 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.655544    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-istio-token" (OuterVolumeSpecName: "istio-token") pod "cb540766-20db-4486-8989-ae0e2b6de57c" (UID: "cb540766-20db-4486-8989-ae0e2b6de57c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:03.754887 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754849    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.754887 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754886    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754900    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754913    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cb540766-20db-4486-8989-ae0e2b6de57c-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754926    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754939    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754951    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cb540766-20db-4486-8989-ae0e2b6de57c-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754963    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cb540766-20db-4486-8989-ae0e2b6de57c-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.755471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.754976    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hdh4\" (UniqueName: \"kubernetes.io/projected/cb540766-20db-4486-8989-ae0e2b6de57c-kube-api-access-2hdh4\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:03.793125 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.793043    2582 generic.go:356] "Generic (PLEG): container finished" podID="cb540766-20db-4486-8989-ae0e2b6de57c" containerID="c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395" exitCode=0
Sep 11 18:05:03.793125 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.793089    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"
Sep 11 18:05:03.793325 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.793130    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" event={"ID":"cb540766-20db-4486-8989-ae0e2b6de57c","Type":"ContainerDied","Data":"c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395"}
Sep 11 18:05:03.793325 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.793185    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5" event={"ID":"cb540766-20db-4486-8989-ae0e2b6de57c","Type":"ContainerDied","Data":"fa14dcf8404746c5f34c8f8277c84d9700b30820c8d87cf0ca143ae43d1fc42f"}
Sep 11 18:05:03.793325 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.793209    2582 scope.go:117] "RemoveContainer" containerID="c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395"
Sep 11 18:05:03.794916 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.794889    2582 generic.go:356] "Generic (PLEG): container finished" podID="ae1578cc-9956-4d6c-bc17-ddce49c36cbd" containerID="5b5156c1af2d71a9606aa703e78b7b5ec3a662f2073382876b65efa31a4eb287" exitCode=0
Sep 11 18:05:03.795042 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.794928    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9" event={"ID":"ae1578cc-9956-4d6c-bc17-ddce49c36cbd","Type":"ContainerDied","Data":"5b5156c1af2d71a9606aa703e78b7b5ec3a662f2073382876b65efa31a4eb287"}
Sep 11 18:05:03.802527 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.802510    2582 scope.go:117] "RemoveContainer" containerID="65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"
Sep 11 18:05:03.810481 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.810458    2582 scope.go:117] "RemoveContainer" containerID="c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395"
Sep 11 18:05:03.810753 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:05:03.810735    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395\": container with ID starting with c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395 not found: ID does not exist" containerID="c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395"
Sep 11 18:05:03.810814 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.810760    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395"} err="failed to get container status \"c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395\": rpc error: code = NotFound desc = could not find container \"c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395\": container with ID starting with c9214d1d0b68fca595ff21c65dd05233228428131fb70a66ab01191e9d91b395 not found: ID does not exist"
Sep 11 18:05:03.810814 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.810776    2582 scope.go:117] "RemoveContainer" containerID="65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"
Sep 11 18:05:03.811044 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:05:03.811026    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022\": container with ID starting with 65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022 not found: ID does not exist" containerID="65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"
Sep 11 18:05:03.811088 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.811050    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022"} err="failed to get container status \"65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022\": rpc error: code = NotFound desc = could not find container \"65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022\": container with ID starting with 65b073c075518c1edf7c47b0dba135c5a93ac10ce194ec4b86f1188f33b66022 not found: ID does not exist"
Sep 11 18:05:03.817081 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.817058    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"]
Sep 11 18:05:03.831920 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:03.831899    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-4ab253-openshift-default-74dbd964db-pg8r5"]
Sep 11 18:05:04.586113 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:04.586080    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:04.939701 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:04.939673    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:05.066937 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.066909    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1578cc-9956-4d6c-bc17-ddce49c36cbd-maas-api-service-ca\") pod \"ae1578cc-9956-4d6c-bc17-ddce49c36cbd\" (UID: \"ae1578cc-9956-4d6c-bc17-ddce49c36cbd\") "
Sep 11 18:05:05.067303 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.067281    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1578cc-9956-4d6c-bc17-ddce49c36cbd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ae1578cc-9956-4d6c-bc17-ddce49c36cbd" (UID: "ae1578cc-9956-4d6c-bc17-ddce49c36cbd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:05:05.168486 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.168400    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1578cc-9956-4d6c-bc17-ddce49c36cbd-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:05.459607 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.459529    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:05.747985 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.747953    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" path="/var/lib/kubelet/pods/cb540766-20db-4486-8989-ae0e2b6de57c/volumes"
Sep 11 18:05:05.808093 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.808065    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"
Sep 11 18:05:05.808252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.808067    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9" event={"ID":"ae1578cc-9956-4d6c-bc17-ddce49c36cbd","Type":"ContainerDied","Data":"07de26d628e8889b719933739adf4bf890c12c6f2e017a7530badf322a5804d9"}
Sep 11 18:05:05.808252 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:05.808185    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="07de26d628e8889b719933739adf4bf890c12c6f2e017a7530badf322a5804d9"
Sep 11 18:05:06.169226 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:06.169126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:12.712856 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:12.712821    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:12.810457 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:12.810423    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:12.907862 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:12.907828    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"]
Sep 11 18:05:12.911814 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:12.911788    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-7f85c3-dhcz9"]
Sep 11 18:05:12.912217 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:12.912185    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:13.008387 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:13.008334    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:13.290610 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:13.290519    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:13.387841 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:13.387809    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:13.748840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:13.748807    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1578cc-9956-4d6c-bc17-ddce49c36cbd" path="/var/lib/kubelet/pods/ae1578cc-9956-4d6c-bc17-ddce49c36cbd/volumes"
Sep 11 18:05:14.173720 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:14.173633    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:15.363223 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:15.363137    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:16.418556 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:16.418522    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:05:20.346964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:20.346917    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:20.445942 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:20.445893    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:20.635466 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:20.635372    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"]
Sep 11 18:05:20.635667 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:20.635642    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" containerID="cri-o://bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09" gracePeriod=30
Sep 11 18:05:20.667457 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:20.667427    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:21.247304 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:21.247260    2582 patch_prober.go:28] interesting pod/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:05:21.247472 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:21.247333    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:05:22.022273 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:22.022235    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:22.394960 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:22.394867    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:22.671892 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:22.671806    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:23.085031 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:23.084991    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:23.770644 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:23.770614    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"]
Sep 11 18:05:23.786154 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:23.786127    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:24.087201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:24.087117    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:24.665642 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:24.665606    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:25.895533 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.895511    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 18:05:25.927513 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.927441    2582 generic.go:356] "Generic (PLEG): container finished" podID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerID="bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09" exitCode=0
Sep 11 18:05:25.927513 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.927485    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"
Sep 11 18:05:25.927717 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.927531    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" event={"ID":"18de25a0-645d-45eb-8cfa-a4ce4f7471e4","Type":"ContainerDied","Data":"bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09"}
Sep 11 18:05:25.927717 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.927592    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr" event={"ID":"18de25a0-645d-45eb-8cfa-a4ce4f7471e4","Type":"ContainerDied","Data":"2d96759ef24843fe94fb43817132044fea6d34ec766426ddc8d7a9dcd25364da"}
Sep 11 18:05:25.927717 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.927627    2582 scope.go:117] "RemoveContainer" containerID="bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09"
Sep 11 18:05:25.938461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.938440    2582 scope.go:117] "RemoveContainer" containerID="2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"
Sep 11 18:05:25.946844 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.946818    2582 scope.go:117] "RemoveContainer" containerID="bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09"
Sep 11 18:05:25.947143 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:05:25.947119    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09\": container with ID starting with bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09 not found: ID does not exist" containerID="bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09"
Sep 11 18:05:25.947226 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.947158    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09"} err="failed to get container status \"bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09\": rpc error: code = NotFound desc = could not find container \"bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09\": container with ID starting with bd149290ae24ed2a7e6eb221b8286eddc8ad97dbbb0986f6ef0767f2af398e09 not found: ID does not exist"
Sep 11 18:05:25.947226 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.947178    2582 scope.go:117] "RemoveContainer" containerID="2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"
Sep 11 18:05:25.947531 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:05:25.947509    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e\": container with ID starting with 2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e not found: ID does not exist" containerID="2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"
Sep 11 18:05:25.947600 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.947536    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e"} err="failed to get container status \"2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e\": rpc error: code = NotFound desc = could not find container \"2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e\": container with ID starting with 2113d25475f9f6f7d43850cf63cf98026ea25a782d343611ccc16c2931bafb4e not found: ID does not exist"
Sep 11 18:05:25.959863 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.959842    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-certs\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.959962 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.959889    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-podinfo\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.959962 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.959916    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istiod-ca-cert\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.959962 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.959945    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-envoy\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.960105 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960071    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-token\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.960160 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960132    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-socket\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.960265 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960146    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:25.960265 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960177    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9cnt\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-kube-api-access-s9cnt\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.960265 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960208    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-credential-socket\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.960459 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960277    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-data\") pod \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\" (UID: \"18de25a0-645d-45eb-8cfa-a4ce4f7471e4\") "
Sep 11 18:05:25.960459 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960399    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:25.960459 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960451    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:05:25.960636 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960616    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:25.960696 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960659    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:25.960798 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960768    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:25.960798 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960787    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:25.960911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960800    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:25.960911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960813    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:25.960911 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.960826    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:25.962463 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.962437    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:25.962569 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.962518    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:05:25.962635 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.962605    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:25.962735 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:25.962703    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-kube-api-access-s9cnt" (OuterVolumeSpecName: "kube-api-access-s9cnt") pod "18de25a0-645d-45eb-8cfa-a4ce4f7471e4" (UID: "18de25a0-645d-45eb-8cfa-a4ce4f7471e4"). InnerVolumeSpecName "kube-api-access-s9cnt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:26.061647 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:26.061614    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:26.061647 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:26.061653    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:26.061818 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:26.061663    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:26.061818 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:26.061671    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9cnt\" (UniqueName: \"kubernetes.io/projected/18de25a0-645d-45eb-8cfa-a4ce4f7471e4-kube-api-access-s9cnt\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:26.252118 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:26.252090    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"]
Sep 11 18:05:26.255386 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:26.255361    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-7f85c3-openshift-default-58586c7c68-7szdr"]
Sep 11 18:05:27.713684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:27.713643    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:05:27.748239 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:27.748195    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" path="/var/lib/kubelet/pods/18de25a0-645d-45eb-8cfa-a4ce4f7471e4/volumes"
Sep 11 18:05:29.759238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:29.759201    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:36.980071 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980039    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"]
Sep 11 18:05:36.980644 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980626    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy"
Sep 11 18:05:36.980714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980647    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy"
Sep 11 18:05:36.980714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980671    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy"
Sep 11 18:05:36.980714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980678    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy"
Sep 11 18:05:36.980714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980707    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy"
Sep 11 18:05:36.980714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980714    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy"
Sep 11 18:05:36.980964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980723    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1578cc-9956-4d6c-bc17-ddce49c36cbd" containerName="revoke-keys"
Sep 11 18:05:36.980964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980728    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1578cc-9956-4d6c-bc17-ddce49c36cbd" containerName="revoke-keys"
Sep 11 18:05:36.980964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980829    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy"
Sep 11 18:05:36.980964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980845    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy"
Sep 11 18:05:36.980964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980855    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy"
Sep 11 18:05:36.980964 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.980864    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1578cc-9956-4d6c-bc17-ddce49c36cbd" containerName="revoke-keys"
Sep 11 18:05:36.985654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.985630    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:36.989410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.989384    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"sub-test-model-a18ac6-kserve-self-signed-certs\""
Sep 11 18:05:36.995658 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:36.995635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"]
Sep 11 18:05:37.008104 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.008078    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:05:37.057220 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.057190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-dshm\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.057394 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.057251    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-tmp-dir\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.057394 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.057325    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt9lr\" (UniqueName: \"kubernetes.io/projected/09774373-72e4-452f-a374-4abfa7da8bdc-kube-api-access-pt9lr\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.057394 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.057372    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09774373-72e4-452f-a374-4abfa7da8bdc-tls-certs\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.057514 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.057418    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-model-cache\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.057514 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.057455    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-home\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158261 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158229    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-tmp-dir\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158454 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158268    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt9lr\" (UniqueName: \"kubernetes.io/projected/09774373-72e4-452f-a374-4abfa7da8bdc-kube-api-access-pt9lr\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158454 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09774373-72e4-452f-a374-4abfa7da8bdc-tls-certs\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158454 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-model-cache\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158454 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-home\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158663 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-dshm\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158712 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158696    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-tmp-dir\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158780 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158760    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-model-cache\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.158842 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.158819    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-home\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.160691 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.160666    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-dshm\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.160875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.160857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09774373-72e4-452f-a374-4abfa7da8bdc-tls-certs\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.170680 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.170659    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt9lr\" (UniqueName: \"kubernetes.io/projected/09774373-72e4-452f-a374-4abfa7da8bdc-kube-api-access-pt9lr\") pod \"sub-test-model-a18ac6-kserve-779bd8d447-4b4bd\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.298006 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.297924    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:37.573006 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.572982    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"]
Sep 11 18:05:37.574727 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:05:37.574702    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod09774373_72e4_452f_a374_4abfa7da8bdc.slice/crio-786d3fff2c60444498014e61110c84ff4664f7aa962f69010fea8620419d06e2 WatchSource:0}: Error finding container 786d3fff2c60444498014e61110c84ff4664f7aa962f69010fea8620419d06e2: Status 404 returned error can't find the container with id 786d3fff2c60444498014e61110c84ff4664f7aa962f69010fea8620419d06e2
Sep 11 18:05:37.999018 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.998988    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd" event={"ID":"09774373-72e4-452f-a374-4abfa7da8bdc","Type":"ContainerStarted","Data":"cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba"}
Sep 11 18:05:37.999621 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:37.999033    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd" event={"ID":"09774373-72e4-452f-a374-4abfa7da8bdc","Type":"ContainerStarted","Data":"786d3fff2c60444498014e61110c84ff4664f7aa962f69010fea8620419d06e2"}
Sep 11 18:05:38.021332 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:38.021290    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd" podStartSLOduration=2.021273591 podStartE2EDuration="2.021273591s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:05:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:05:38.019295233 +0000 UTC m=+1620.889338177" watchObservedRunningTime="2026-09-11 18:05:38.021273591 +0000 UTC m=+1620.891316537"
Sep 11 18:05:38.790527 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:38.790466    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" podUID="e1228f2f-278a-4423-94b5-28beb41ef2a4" containerName="main" containerID="cri-o://14335e87942cf05e565f75bb7c05a82e512b7d55899aa5e6e04db87a02d3408e" gracePeriod=45
Sep 11 18:05:38.887558 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:38.887513    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" podUID="e1228f2f-278a-4423-94b5-28beb41ef2a4" containerName="main" probeResult="failure" output="Get \"https://10.132.0.74:8000/ready\": dial tcp 10.132.0.74:8000: connect: connection refused"
Sep 11 18:05:39.005932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.005900    2582 generic.go:356] "Generic (PLEG): container finished" podID="e1228f2f-278a-4423-94b5-28beb41ef2a4" containerID="14335e87942cf05e565f75bb7c05a82e512b7d55899aa5e6e04db87a02d3408e" exitCode=0
Sep 11 18:05:39.006299 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.005962    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" event={"ID":"e1228f2f-278a-4423-94b5-28beb41ef2a4","Type":"ContainerDied","Data":"14335e87942cf05e565f75bb7c05a82e512b7d55899aa5e6e04db87a02d3408e"}
Sep 11 18:05:39.069711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.069683    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 18:05:39.176482 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176444    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1228f2f-278a-4423-94b5-28beb41ef2a4-tls-certs\") pod \"e1228f2f-278a-4423-94b5-28beb41ef2a4\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") "
Sep 11 18:05:39.176654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176546    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-dshm\") pod \"e1228f2f-278a-4423-94b5-28beb41ef2a4\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") "
Sep 11 18:05:39.176654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176576    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-tmp-dir\") pod \"e1228f2f-278a-4423-94b5-28beb41ef2a4\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") "
Sep 11 18:05:39.176654 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176619    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-model-cache\") pod \"e1228f2f-278a-4423-94b5-28beb41ef2a4\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") "
Sep 11 18:05:39.176838 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176676    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkznd\" (UniqueName: \"kubernetes.io/projected/e1228f2f-278a-4423-94b5-28beb41ef2a4-kube-api-access-zkznd\") pod \"e1228f2f-278a-4423-94b5-28beb41ef2a4\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") "
Sep 11 18:05:39.176838 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176728    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-home\") pod \"e1228f2f-278a-4423-94b5-28beb41ef2a4\" (UID: \"e1228f2f-278a-4423-94b5-28beb41ef2a4\") "
Sep 11 18:05:39.176941 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176870    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e1228f2f-278a-4423-94b5-28beb41ef2a4" (UID: "e1228f2f-278a-4423-94b5-28beb41ef2a4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:39.176941 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.176893    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-model-cache" (OuterVolumeSpecName: "model-cache") pod "e1228f2f-278a-4423-94b5-28beb41ef2a4" (UID: "e1228f2f-278a-4423-94b5-28beb41ef2a4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:39.177145 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.177071    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-home" (OuterVolumeSpecName: "home") pod "e1228f2f-278a-4423-94b5-28beb41ef2a4" (UID: "e1228f2f-278a-4423-94b5-28beb41ef2a4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:39.177145 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.177084    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:39.177145 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.177121    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:39.178875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.178850    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-dshm" (OuterVolumeSpecName: "dshm") pod "e1228f2f-278a-4423-94b5-28beb41ef2a4" (UID: "e1228f2f-278a-4423-94b5-28beb41ef2a4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:05:39.178875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.178862    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1228f2f-278a-4423-94b5-28beb41ef2a4-kube-api-access-zkznd" (OuterVolumeSpecName: "kube-api-access-zkznd") pod "e1228f2f-278a-4423-94b5-28beb41ef2a4" (UID: "e1228f2f-278a-4423-94b5-28beb41ef2a4"). InnerVolumeSpecName "kube-api-access-zkznd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:05:39.179010 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.178880    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1228f2f-278a-4423-94b5-28beb41ef2a4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e1228f2f-278a-4423-94b5-28beb41ef2a4" (UID: "e1228f2f-278a-4423-94b5-28beb41ef2a4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:05:39.278244 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.278208    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:39.278244 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.278237    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkznd\" (UniqueName: \"kubernetes.io/projected/e1228f2f-278a-4423-94b5-28beb41ef2a4-kube-api-access-zkznd\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:39.278244 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.278248    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1228f2f-278a-4423-94b5-28beb41ef2a4-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:39.278491 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:39.278257    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1228f2f-278a-4423-94b5-28beb41ef2a4-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:05:40.013053 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:40.012972    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"
Sep 11 18:05:40.013472 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:40.012971    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x" event={"ID":"e1228f2f-278a-4423-94b5-28beb41ef2a4","Type":"ContainerDied","Data":"158567e302586719f15fd3ff8651d2c4c4b7d2fe77437f4807978281c551a438"}
Sep 11 18:05:40.013472 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:40.013101    2582 scope.go:117] "RemoveContainer" containerID="14335e87942cf05e565f75bb7c05a82e512b7d55899aa5e6e04db87a02d3408e"
Sep 11 18:05:40.044385 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:40.044158    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"]
Sep 11 18:05:40.046268 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:40.046241    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/auth-test-model-a18ac6-kserve-7dc5cf95bc-4sv9x"]
Sep 11 18:05:40.074540 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:40.074516    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:05:41.747475 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:41.747441    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1228f2f-278a-4423-94b5-28beb41ef2a4" path="/var/lib/kubelet/pods/e1228f2f-278a-4423-94b5-28beb41ef2a4/volumes"
Sep 11 18:05:47.298486 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:47.298442    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:47.298864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:47.298513    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:47.311302 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:47.311277    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:48.068283 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:48.068252    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:05:59.629470 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:05:59.629431    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:06:17.989506 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.989469    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"]
Sep 11 18:06:17.990085 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.990065    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy"
Sep 11 18:06:17.990129 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.990090    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="18de25a0-645d-45eb-8cfa-a4ce4f7471e4" containerName="istio-proxy"
Sep 11 18:06:17.990129 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.990126    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1228f2f-278a-4423-94b5-28beb41ef2a4" containerName="main"
Sep 11 18:06:17.990195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.990135    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1228f2f-278a-4423-94b5-28beb41ef2a4" containerName="main"
Sep 11 18:06:17.990282 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.990268    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1228f2f-278a-4423-94b5-28beb41ef2a4" containerName="main"
Sep 11 18:06:17.990317 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.990293    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb540766-20db-4486-8989-ae0e2b6de57c" containerName="istio-proxy"
Sep 11 18:06:17.993543 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.993526    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:17.996831 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.996809    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"sub-test-model-1bf937-kserve-self-signed-certs\""
Sep 11 18:06:17.996831 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.996828    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"openshift-service-ca.crt\""
Sep 11 18:06:17.997017 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.996810    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"kube-root-ca.crt\""
Sep 11 18:06:17.997017 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:17.996810    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"default-dockercfg-8p6ch\""
Sep 11 18:06:18.005797 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.005774    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"]
Sep 11 18:06:18.037404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.037373    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:06:18.101187 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.101163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89qcr\" (UniqueName: \"kubernetes.io/projected/3abc8790-a46e-4962-ad35-c21a03a8f5e0-kube-api-access-89qcr\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.101314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.101197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-home\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.101314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.101216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tmp-dir\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.101314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.101293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-model-cache\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.101484 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.101370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-dshm\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.101484 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.101420    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tls-certs\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202191 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202157    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-model-cache\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202381 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202207    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-dshm\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202381 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202303    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tls-certs\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202508 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202483    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89qcr\" (UniqueName: \"kubernetes.io/projected/3abc8790-a46e-4962-ad35-c21a03a8f5e0-kube-api-access-89qcr\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202557 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202544    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-home\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202641 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202617    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-model-cache\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202806 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202653    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tmp-dir\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202882 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202800    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-home\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.202931 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.202916    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tmp-dir\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.204612 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.204591    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-dshm\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.204875 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.204858    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tls-certs\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.214431 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.214408    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89qcr\" (UniqueName: \"kubernetes.io/projected/3abc8790-a46e-4962-ad35-c21a03a8f5e0-kube-api-access-89qcr\") pod \"sub-test-model-1bf937-kserve-5df87b6b6c-x5clr\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.303074 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.302993    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:18.585929 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:18.585892    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"]
Sep 11 18:06:18.588287 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:06:18.588264    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3abc8790_a46e_4962_ad35_c21a03a8f5e0.slice/crio-e9c6a36f429ef680e3ad2a0baa01ba4c25e6302ca069bcd141d1bf3e62e64d68 WatchSource:0}: Error finding container e9c6a36f429ef680e3ad2a0baa01ba4c25e6302ca069bcd141d1bf3e62e64d68: Status 404 returned error can't find the container with id e9c6a36f429ef680e3ad2a0baa01ba4c25e6302ca069bcd141d1bf3e62e64d68
Sep 11 18:06:19.247386 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:19.247339    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr" event={"ID":"3abc8790-a46e-4962-ad35-c21a03a8f5e0","Type":"ContainerStarted","Data":"ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32"}
Sep 11 18:06:19.247852 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:19.247401    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr" event={"ID":"3abc8790-a46e-4962-ad35-c21a03a8f5e0","Type":"ContainerStarted","Data":"e9c6a36f429ef680e3ad2a0baa01ba4c25e6302ca069bcd141d1bf3e62e64d68"}
Sep 11 18:06:19.281341 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:19.281290    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr" podStartSLOduration=2.281275158 podStartE2EDuration="2.281275158s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:06:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:06:19.279823861 +0000 UTC m=+1662.149866807" watchObservedRunningTime="2026-09-11 18:06:19.281275158 +0000 UTC m=+1662.151318102"
Sep 11 18:06:21.670783 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:21.670741    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:06:28.304581 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:28.304535    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:28.304988 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:28.304598    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:28.316939 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:28.316916    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:29.316850 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:29.316818    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:06:38.017256 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:38.017221    2582 scope.go:117] "RemoveContainer" containerID="5c898e937dcec56a3160b4297100bb512c872f10ee21917c0e27be65f167d611"
Sep 11 18:06:38.025815 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:38.025796    2582 scope.go:117] "RemoveContainer" containerID="7cb67dcbedcadd05fe85a9cd669786cbc69813e0f5ab7360374b3e356eb40df7"
Sep 11 18:06:38.033444 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:38.033420    2582 scope.go:117] "RemoveContainer" containerID="c5810857964602e5d2b2b12e74d7caaba247aff3347a01d09db4d98ee529c7ff"
Sep 11 18:06:38.040728 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:38.040708    2582 scope.go:117] "RemoveContainer" containerID="d1e81343ea2be5d00567d96952e3720ec5ef3f923b345e16023e192f19a03fca"
Sep 11 18:06:38.421470 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:38.421390    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:06:53.605113 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:53.605083    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:06:55.134195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:55.134163    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"]
Sep 11 18:06:55.151739 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:55.151712    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:06:56.189520 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:56.189490    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"]
Sep 11 18:06:56.212805 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:06:56.212778    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:07:04.826870 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:04.826790    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:07:10.153363 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.153309    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd" podUID="09774373-72e4-452f-a374-4abfa7da8bdc" containerName="main" containerID="cri-o://cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba" gracePeriod=45
Sep 11 18:07:10.404263 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.404206    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:07:10.482100 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482073    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-model-cache\") pod \"09774373-72e4-452f-a374-4abfa7da8bdc\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") "
Sep 11 18:07:10.482100 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482104    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-dshm\") pod \"09774373-72e4-452f-a374-4abfa7da8bdc\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") "
Sep 11 18:07:10.482318 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482134    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09774373-72e4-452f-a374-4abfa7da8bdc-tls-certs\") pod \"09774373-72e4-452f-a374-4abfa7da8bdc\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") "
Sep 11 18:07:10.482318 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482179    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt9lr\" (UniqueName: \"kubernetes.io/projected/09774373-72e4-452f-a374-4abfa7da8bdc-kube-api-access-pt9lr\") pod \"09774373-72e4-452f-a374-4abfa7da8bdc\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") "
Sep 11 18:07:10.482318 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482235    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-home\") pod \"09774373-72e4-452f-a374-4abfa7da8bdc\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") "
Sep 11 18:07:10.482318 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482272    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-tmp-dir\") pod \"09774373-72e4-452f-a374-4abfa7da8bdc\" (UID: \"09774373-72e4-452f-a374-4abfa7da8bdc\") "
Sep 11 18:07:10.482558 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482398    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-model-cache" (OuterVolumeSpecName: "model-cache") pod "09774373-72e4-452f-a374-4abfa7da8bdc" (UID: "09774373-72e4-452f-a374-4abfa7da8bdc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:10.482558 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482511    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-home" (OuterVolumeSpecName: "home") pod "09774373-72e4-452f-a374-4abfa7da8bdc" (UID: "09774373-72e4-452f-a374-4abfa7da8bdc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:10.482664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482594    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "09774373-72e4-452f-a374-4abfa7da8bdc" (UID: "09774373-72e4-452f-a374-4abfa7da8bdc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:10.482711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482664    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:10.482711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.482684    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:10.484285 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.484257    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-dshm" (OuterVolumeSpecName: "dshm") pod "09774373-72e4-452f-a374-4abfa7da8bdc" (UID: "09774373-72e4-452f-a374-4abfa7da8bdc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:10.484408 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.484302    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09774373-72e4-452f-a374-4abfa7da8bdc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "09774373-72e4-452f-a374-4abfa7da8bdc" (UID: "09774373-72e4-452f-a374-4abfa7da8bdc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:07:10.484408 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.484364    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09774373-72e4-452f-a374-4abfa7da8bdc-kube-api-access-pt9lr" (OuterVolumeSpecName: "kube-api-access-pt9lr") pod "09774373-72e4-452f-a374-4abfa7da8bdc" (UID: "09774373-72e4-452f-a374-4abfa7da8bdc"). InnerVolumeSpecName "kube-api-access-pt9lr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:07:10.534674 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.534645    2582 generic.go:356] "Generic (PLEG): container finished" podID="09774373-72e4-452f-a374-4abfa7da8bdc" containerID="cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba" exitCode=0
Sep 11 18:07:10.534840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.534694    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"
Sep 11 18:07:10.534840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.534731    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd" event={"ID":"09774373-72e4-452f-a374-4abfa7da8bdc","Type":"ContainerDied","Data":"cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba"}
Sep 11 18:07:10.534840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.534774    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd" event={"ID":"09774373-72e4-452f-a374-4abfa7da8bdc","Type":"ContainerDied","Data":"786d3fff2c60444498014e61110c84ff4664f7aa962f69010fea8620419d06e2"}
Sep 11 18:07:10.534840 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.534792    2582 scope.go:117] "RemoveContainer" containerID="cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba"
Sep 11 18:07:10.544328 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.544312    2582 scope.go:117] "RemoveContainer" containerID="cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba"
Sep 11 18:07:10.544620 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:07:10.544597    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba\": container with ID starting with cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba not found: ID does not exist" containerID="cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba"
Sep 11 18:07:10.544675 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.544627    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba"} err="failed to get container status \"cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba\": rpc error: code = NotFound desc = could not find container \"cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba\": container with ID starting with cfd114be0846535acdc40c44af47b9451b457e36ca0713ba4657c53b2f8139ba not found: ID does not exist"
Sep 11 18:07:10.559743 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.559712    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"]
Sep 11 18:07:10.565141 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.565120    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/sub-test-model-a18ac6-kserve-779bd8d447-4b4bd"]
Sep 11 18:07:10.583942 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.583920    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:10.583942 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.583940    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/09774373-72e4-452f-a374-4abfa7da8bdc-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:10.584053 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.583950    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt9lr\" (UniqueName: \"kubernetes.io/projected/09774373-72e4-452f-a374-4abfa7da8bdc-kube-api-access-pt9lr\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:10.584053 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:10.583959    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/09774373-72e4-452f-a374-4abfa7da8bdc-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.208474 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.208421    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr" podUID="3abc8790-a46e-4962-ad35-c21a03a8f5e0" containerName="main" containerID="cri-o://ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32" gracePeriod=45
Sep 11 18:07:11.465461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.465400    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:07:11.541245 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.541214    2582 generic.go:356] "Generic (PLEG): container finished" podID="3abc8790-a46e-4962-ad35-c21a03a8f5e0" containerID="ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32" exitCode=0
Sep 11 18:07:11.541428 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.541253    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"
Sep 11 18:07:11.541428 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.541303    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr" event={"ID":"3abc8790-a46e-4962-ad35-c21a03a8f5e0","Type":"ContainerDied","Data":"ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32"}
Sep 11 18:07:11.541428 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.541361    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr" event={"ID":"3abc8790-a46e-4962-ad35-c21a03a8f5e0","Type":"ContainerDied","Data":"e9c6a36f429ef680e3ad2a0baa01ba4c25e6302ca069bcd141d1bf3e62e64d68"}
Sep 11 18:07:11.541428 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.541385    2582 scope.go:117] "RemoveContainer" containerID="ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32"
Sep 11 18:07:11.549857 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.549839    2582 scope.go:117] "RemoveContainer" containerID="ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32"
Sep 11 18:07:11.550128 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:07:11.550106    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32\": container with ID starting with ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32 not found: ID does not exist" containerID="ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32"
Sep 11 18:07:11.550164 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.550140    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32"} err="failed to get container status \"ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32\": rpc error: code = NotFound desc = could not find container \"ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32\": container with ID starting with ca1831648b3218257c160fee5c8e8005573c435b8fa239b47d4245365906da32 not found: ID does not exist"
Sep 11 18:07:11.593731 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593708    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-dshm\") pod \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") "
Sep 11 18:07:11.593802 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593739    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tmp-dir\") pod \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") "
Sep 11 18:07:11.593802 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593787    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tls-certs\") pod \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") "
Sep 11 18:07:11.593880 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593826    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-home\") pod \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") "
Sep 11 18:07:11.593914 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593898    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89qcr\" (UniqueName: \"kubernetes.io/projected/3abc8790-a46e-4962-ad35-c21a03a8f5e0-kube-api-access-89qcr\") pod \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") "
Sep 11 18:07:11.593965 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593925    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-model-cache\") pod \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\" (UID: \"3abc8790-a46e-4962-ad35-c21a03a8f5e0\") "
Sep 11 18:07:11.594021 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.593984    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3abc8790-a46e-4962-ad35-c21a03a8f5e0" (UID: "3abc8790-a46e-4962-ad35-c21a03a8f5e0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:11.594094 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.594070    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-home" (OuterVolumeSpecName: "home") pod "3abc8790-a46e-4962-ad35-c21a03a8f5e0" (UID: "3abc8790-a46e-4962-ad35-c21a03a8f5e0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:11.594188 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.594168    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-model-cache" (OuterVolumeSpecName: "model-cache") pod "3abc8790-a46e-4962-ad35-c21a03a8f5e0" (UID: "3abc8790-a46e-4962-ad35-c21a03a8f5e0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:11.594289 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.594271    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.594331 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.594296    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.594331 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.594309    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.595967 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.595941    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-dshm" (OuterVolumeSpecName: "dshm") pod "3abc8790-a46e-4962-ad35-c21a03a8f5e0" (UID: "3abc8790-a46e-4962-ad35-c21a03a8f5e0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:07:11.596049 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.596030    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3abc8790-a46e-4962-ad35-c21a03a8f5e0" (UID: "3abc8790-a46e-4962-ad35-c21a03a8f5e0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:07:11.596090 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.596029    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3abc8790-a46e-4962-ad35-c21a03a8f5e0-kube-api-access-89qcr" (OuterVolumeSpecName: "kube-api-access-89qcr") pod "3abc8790-a46e-4962-ad35-c21a03a8f5e0" (UID: "3abc8790-a46e-4962-ad35-c21a03a8f5e0"). InnerVolumeSpecName "kube-api-access-89qcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:07:11.695370 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.695317    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3abc8790-a46e-4962-ad35-c21a03a8f5e0-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.695501 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.695374    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3abc8790-a46e-4962-ad35-c21a03a8f5e0-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.695501 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.695392    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89qcr\" (UniqueName: \"kubernetes.io/projected/3abc8790-a46e-4962-ad35-c21a03a8f5e0-kube-api-access-89qcr\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:07:11.747205 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.747174    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09774373-72e4-452f-a374-4abfa7da8bdc" path="/var/lib/kubelet/pods/09774373-72e4-452f-a374-4abfa7da8bdc/volumes"
Sep 11 18:07:11.863255 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.863221    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"]
Sep 11 18:07:11.872549 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:11.872523    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/sub-test-model-1bf937-kserve-5df87b6b6c-x5clr"]
Sep 11 18:07:13.746646 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:13.746611    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3abc8790-a46e-4962-ad35-c21a03a8f5e0" path="/var/lib/kubelet/pods/3abc8790-a46e-4962-ad35-c21a03a8f5e0/volumes"
Sep 11 18:07:18.445197 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445162    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"]
Sep 11 18:07:18.445590 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445568    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3abc8790-a46e-4962-ad35-c21a03a8f5e0" containerName="main"
Sep 11 18:07:18.445590 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445579    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3abc8790-a46e-4962-ad35-c21a03a8f5e0" containerName="main"
Sep 11 18:07:18.445590 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445590    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09774373-72e4-452f-a374-4abfa7da8bdc" containerName="main"
Sep 11 18:07:18.445693 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445596    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="09774373-72e4-452f-a374-4abfa7da8bdc" containerName="main"
Sep 11 18:07:18.445693 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445684    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3abc8790-a46e-4962-ad35-c21a03a8f5e0" containerName="main"
Sep 11 18:07:18.445762 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.445694    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="09774373-72e4-452f-a374-4abfa7da8bdc" containerName="main"
Sep 11 18:07:18.450020 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.450002    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.457532 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.457505    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"default-dockercfg-q7pct\""
Sep 11 18:07:18.457660 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.457513    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"rate-test-model-a18ac6-kserve-self-signed-certs\""
Sep 11 18:07:18.457660 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.457604    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"openshift-service-ca.crt\""
Sep 11 18:07:18.457660 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.457605    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-a18ac6\"/\"kube-root-ca.crt\""
Sep 11 18:07:18.484466 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.484435    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"]
Sep 11 18:07:18.539055 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.539025    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:07:18.553127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.553101    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q46jf\" (UniqueName: \"kubernetes.io/projected/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-kube-api-access-q46jf\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.553238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.553156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-dshm\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.553238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.553213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tls-certs\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.553238 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.553229    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-model-cache\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.553380 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.553245    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-home\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.553380 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.553262    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tmp-dir\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.654577 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.654548    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-home\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.654707 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.654583    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tmp-dir\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.654707 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.654640    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q46jf\" (UniqueName: \"kubernetes.io/projected/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-kube-api-access-q46jf\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.654707 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.654679    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-dshm\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.654707 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.654703    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tls-certs\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.654930 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.654721    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-model-cache\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.655032 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.655009    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-home\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.655032 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.655026    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tmp-dir\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.655153 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.655137    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-model-cache\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.656948 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.656927    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-dshm\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.657341 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.657320    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tls-certs\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.667292 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.667263    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q46jf\" (UniqueName: \"kubernetes.io/projected/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-kube-api-access-q46jf\") pod \"rate-test-model-a18ac6-kserve-6885bf77cf-56g4z\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") " pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:18.758931 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:18.758894    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:19.043074 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:19.043039    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"]
Sep 11 18:07:19.044568 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:07:19.044542    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6bf807a9_e755_4d4d_b61e_35f3a8c44a02.slice/crio-1e874de9845121021bf6aa7fde23548d88ad0db17ae41072c2b18e14c95eff80 WatchSource:0}: Error finding container 1e874de9845121021bf6aa7fde23548d88ad0db17ae41072c2b18e14c95eff80: Status 404 returned error can't find the container with id 1e874de9845121021bf6aa7fde23548d88ad0db17ae41072c2b18e14c95eff80
Sep 11 18:07:19.585995 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:19.585960    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z" event={"ID":"6bf807a9-e755-4d4d-b61e-35f3a8c44a02","Type":"ContainerStarted","Data":"ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830"}
Sep 11 18:07:19.586410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:19.586003    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z" event={"ID":"6bf807a9-e755-4d4d-b61e-35f3a8c44a02","Type":"ContainerStarted","Data":"1e874de9845121021bf6aa7fde23548d88ad0db17ae41072c2b18e14c95eff80"}
Sep 11 18:07:19.620936 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:19.620881    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z" podStartSLOduration=1.620863824 podStartE2EDuration="1.620863824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:07:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:07:19.620003571 +0000 UTC m=+1722.490046517" watchObservedRunningTime="2026-09-11 18:07:19.620863824 +0000 UTC m=+1722.490906768"
Sep 11 18:07:21.854954 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:21.854920    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:07:28.759465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:28.759422    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:28.759465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:28.759471    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:28.772199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:28.772172    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:07:29.649612 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:07:29.649582    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:08:01.093527 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.093489    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"]
Sep 11 18:08:01.097193 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.097170    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.100421 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.100398    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"kube-root-ca.crt\""
Sep 11 18:08:01.100553 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.100427    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"default-dockercfg-8p6ch\""
Sep 11 18:08:01.100553 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.100426    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"openshift-service-ca.crt\""
Sep 11 18:08:01.101181 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.101157    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-1bf937\"/\"rate-test-model-1bf937-kserve-self-signed-certs\""
Sep 11 18:08:01.110913 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.110888    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"]
Sep 11 18:08:01.138624 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.138601    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:01.228213 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.228175    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-model-cache\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.228213 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.228213    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f70f086-8283-46f1-8af6-3a4e18367a19-tls-certs\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.228511 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.228245    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-tmp-dir\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.228511 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.228313    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4bmd\" (UniqueName: \"kubernetes.io/projected/9f70f086-8283-46f1-8af6-3a4e18367a19-kube-api-access-r4bmd\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.228511 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.228411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-home\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.228511 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.228453    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-dshm\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.329786 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.329747    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-home\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.329968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.329797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-dshm\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.329968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.329855    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-model-cache\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.329968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.329878    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f70f086-8283-46f1-8af6-3a4e18367a19-tls-certs\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.329968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.329910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-tmp-dir\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.329968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.329955    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4bmd\" (UniqueName: \"kubernetes.io/projected/9f70f086-8283-46f1-8af6-3a4e18367a19-kube-api-access-r4bmd\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.330237 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.330216    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-model-cache\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.330298 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.330214    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-home\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.330379 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.330289    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-tmp-dir\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.332166 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.332141    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-dshm\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.332470 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.332451    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f70f086-8283-46f1-8af6-3a4e18367a19-tls-certs\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.342938 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.342907    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4bmd\" (UniqueName: \"kubernetes.io/projected/9f70f086-8283-46f1-8af6-3a4e18367a19-kube-api-access-r4bmd\") pod \"rate-test-model-1bf937-kserve-7c76d64d47-br82d\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") " pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.406838 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.406756    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:01.682501 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.682475    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"]
Sep 11 18:08:01.685230 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:08:01.685200    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f70f086_8283_46f1_8af6_3a4e18367a19.slice/crio-c73360ebd783f5d9ddf2b52fc4a6731cf2af0d73986503be405df0164fb4746f WatchSource:0}: Error finding container c73360ebd783f5d9ddf2b52fc4a6731cf2af0d73986503be405df0164fb4746f: Status 404 returned error can't find the container with id c73360ebd783f5d9ddf2b52fc4a6731cf2af0d73986503be405df0164fb4746f
Sep 11 18:08:01.821137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.821103    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d" event={"ID":"9f70f086-8283-46f1-8af6-3a4e18367a19","Type":"ContainerStarted","Data":"0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118"}
Sep 11 18:08:01.821137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.821144    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d" event={"ID":"9f70f086-8283-46f1-8af6-3a4e18367a19","Type":"ContainerStarted","Data":"c73360ebd783f5d9ddf2b52fc4a6731cf2af0d73986503be405df0164fb4746f"}
Sep 11 18:08:01.840011 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:01.839955    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d" podStartSLOduration=0.839939366 podStartE2EDuration="839.939366ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:08:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:08:01.838221936 +0000 UTC m=+1764.708264882" watchObservedRunningTime="2026-09-11 18:08:01.839939366 +0000 UTC m=+1764.709982311"
Sep 11 18:08:04.616895 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:04.616858    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:08:11.407872 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:11.407818    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:11.408309 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:11.407936    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:11.420724 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:11.420688    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:11.887962 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:11.887932    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:21.511012 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:21.510975    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:08:23.829462 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:23.829414    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:30.404816 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:30.404737    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:08:32.375259 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:32.375228    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:32.451854 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:32.451826    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:36.151685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:36.151591    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"]
Sep 11 18:08:36.175494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:36.175460    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:36.970482 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:36.970449    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"]
Sep 11 18:08:36.984861 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:36.984832    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:37.441634 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:37.441603    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:37.732251 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:37.732213    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:08:37.738050 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:37.738023    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:08:37.908750 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:37.908715    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:38.153973 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:38.153884    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:41.566048 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:41.566016    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:41.715624 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:41.715597    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:41.822560 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:41.822474    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:42.782515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:42.782487    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:43.698987 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:43.698952    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:43.798486 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:43.798443    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:43.910304 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:43.910273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:44.281732 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:44.281693    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:44.388882 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:44.388844    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:08:45.160834 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:45.160796    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:08:51.170481 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.170445    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z" podUID="6bf807a9-e755-4d4d-b61e-35f3a8c44a02" containerName="main" containerID="cri-o://ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830" gracePeriod=45
Sep 11 18:08:51.418969 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.418942    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:08:51.592666 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592631    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-dshm\") pod \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") "
Sep 11 18:08:51.592825 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592679    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-model-cache\") pod \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") "
Sep 11 18:08:51.592825 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592746    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q46jf\" (UniqueName: \"kubernetes.io/projected/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-kube-api-access-q46jf\") pod \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") "
Sep 11 18:08:51.592825 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592775    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tls-certs\") pod \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") "
Sep 11 18:08:51.593023 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592837    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tmp-dir\") pod \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") "
Sep 11 18:08:51.593023 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592878    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-home\") pod \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\" (UID: \"6bf807a9-e755-4d4d-b61e-35f3a8c44a02\") "
Sep 11 18:08:51.593023 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.592997    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-model-cache" (OuterVolumeSpecName: "model-cache") pod "6bf807a9-e755-4d4d-b61e-35f3a8c44a02" (UID: "6bf807a9-e755-4d4d-b61e-35f3a8c44a02"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:51.593178 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.593110    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6bf807a9-e755-4d4d-b61e-35f3a8c44a02" (UID: "6bf807a9-e755-4d4d-b61e-35f3a8c44a02"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:51.593178 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.593168    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:51.593297 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.593175    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-home" (OuterVolumeSpecName: "home") pod "6bf807a9-e755-4d4d-b61e-35f3a8c44a02" (UID: "6bf807a9-e755-4d4d-b61e-35f3a8c44a02"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:51.594785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.594762    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-dshm" (OuterVolumeSpecName: "dshm") pod "6bf807a9-e755-4d4d-b61e-35f3a8c44a02" (UID: "6bf807a9-e755-4d4d-b61e-35f3a8c44a02"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:51.594889 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.594815    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-kube-api-access-q46jf" (OuterVolumeSpecName: "kube-api-access-q46jf") pod "6bf807a9-e755-4d4d-b61e-35f3a8c44a02" (UID: "6bf807a9-e755-4d4d-b61e-35f3a8c44a02"). InnerVolumeSpecName "kube-api-access-q46jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:08:51.594889 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.594876    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6bf807a9-e755-4d4d-b61e-35f3a8c44a02" (UID: "6bf807a9-e755-4d4d-b61e-35f3a8c44a02"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:08:51.694412 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.694382    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:51.694412 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.694407    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:51.694412 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.694416    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:51.694664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.694425    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q46jf\" (UniqueName: \"kubernetes.io/projected/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-kube-api-access-q46jf\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:51.694664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.694435    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6bf807a9-e755-4d4d-b61e-35f3a8c44a02-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:51.989436 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:51.989378    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d" podUID="9f70f086-8283-46f1-8af6-3a4e18367a19" containerName="main" containerID="cri-o://0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118" gracePeriod=45
Sep 11 18:08:52.095207 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.095170    2582 generic.go:356] "Generic (PLEG): container finished" podID="6bf807a9-e755-4d4d-b61e-35f3a8c44a02" containerID="ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830" exitCode=0
Sep 11 18:08:52.095395 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.095230    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"
Sep 11 18:08:52.095395 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.095256    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z" event={"ID":"6bf807a9-e755-4d4d-b61e-35f3a8c44a02","Type":"ContainerDied","Data":"ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830"}
Sep 11 18:08:52.095395 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.095309    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z" event={"ID":"6bf807a9-e755-4d4d-b61e-35f3a8c44a02","Type":"ContainerDied","Data":"1e874de9845121021bf6aa7fde23548d88ad0db17ae41072c2b18e14c95eff80"}
Sep 11 18:08:52.095395 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.095332    2582 scope.go:117] "RemoveContainer" containerID="ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830"
Sep 11 18:08:52.106887 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.106863    2582 scope.go:117] "RemoveContainer" containerID="ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830"
Sep 11 18:08:52.107170 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:08:52.107144    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830\": container with ID starting with ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830 not found: ID does not exist" containerID="ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830"
Sep 11 18:08:52.107267 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.107179    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830"} err="failed to get container status \"ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830\": rpc error: code = NotFound desc = could not find container \"ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830\": container with ID starting with ca896a869a7b2b9867af5fb4cfacac0c9edce031115b94b46d7fe46399479830 not found: ID does not exist"
Sep 11 18:08:52.119756 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.119731    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"]
Sep 11 18:08:52.132960 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.132935    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-a18ac6/rate-test-model-a18ac6-kserve-6885bf77cf-56g4z"]
Sep 11 18:08:52.242431 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.242340    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:52.400930 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.400895    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-model-cache\") pod \"9f70f086-8283-46f1-8af6-3a4e18367a19\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") "
Sep 11 18:08:52.401101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.400944    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-home\") pod \"9f70f086-8283-46f1-8af6-3a4e18367a19\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") "
Sep 11 18:08:52.401101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.400977    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r4bmd\" (UniqueName: \"kubernetes.io/projected/9f70f086-8283-46f1-8af6-3a4e18367a19-kube-api-access-r4bmd\") pod \"9f70f086-8283-46f1-8af6-3a4e18367a19\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") "
Sep 11 18:08:52.401101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.400995    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-dshm\") pod \"9f70f086-8283-46f1-8af6-3a4e18367a19\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") "
Sep 11 18:08:52.401101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.401013    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f70f086-8283-46f1-8af6-3a4e18367a19-tls-certs\") pod \"9f70f086-8283-46f1-8af6-3a4e18367a19\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") "
Sep 11 18:08:52.401101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.401040    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-tmp-dir\") pod \"9f70f086-8283-46f1-8af6-3a4e18367a19\" (UID: \"9f70f086-8283-46f1-8af6-3a4e18367a19\") "
Sep 11 18:08:52.401423 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.401208    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f70f086-8283-46f1-8af6-3a4e18367a19" (UID: "9f70f086-8283-46f1-8af6-3a4e18367a19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:52.401423 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.401329    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:52.401423 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.401307    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-home" (OuterVolumeSpecName: "home") pod "9f70f086-8283-46f1-8af6-3a4e18367a19" (UID: "9f70f086-8283-46f1-8af6-3a4e18367a19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:52.401423 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.401385    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f70f086-8283-46f1-8af6-3a4e18367a19" (UID: "9f70f086-8283-46f1-8af6-3a4e18367a19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:52.403242 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.403219    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-dshm" (OuterVolumeSpecName: "dshm") pod "9f70f086-8283-46f1-8af6-3a4e18367a19" (UID: "9f70f086-8283-46f1-8af6-3a4e18367a19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:08:52.403329 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.403223    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f70f086-8283-46f1-8af6-3a4e18367a19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f70f086-8283-46f1-8af6-3a4e18367a19" (UID: "9f70f086-8283-46f1-8af6-3a4e18367a19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:08:52.403329 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.403255    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f70f086-8283-46f1-8af6-3a4e18367a19-kube-api-access-r4bmd" (OuterVolumeSpecName: "kube-api-access-r4bmd") pod "9f70f086-8283-46f1-8af6-3a4e18367a19" (UID: "9f70f086-8283-46f1-8af6-3a4e18367a19"). InnerVolumeSpecName "kube-api-access-r4bmd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:08:52.502567 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.502498    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:52.502567 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.502522    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4bmd\" (UniqueName: \"kubernetes.io/projected/9f70f086-8283-46f1-8af6-3a4e18367a19-kube-api-access-r4bmd\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:52.502567 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.502534    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:52.502567 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.502542    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f70f086-8283-46f1-8af6-3a4e18367a19-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:52.502567 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:52.502550    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f70f086-8283-46f1-8af6-3a4e18367a19-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:08:53.102113 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.102077    2582 generic.go:356] "Generic (PLEG): container finished" podID="9f70f086-8283-46f1-8af6-3a4e18367a19" containerID="0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118" exitCode=0
Sep 11 18:08:53.102310 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.102117    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"
Sep 11 18:08:53.102310 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.102168    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d" event={"ID":"9f70f086-8283-46f1-8af6-3a4e18367a19","Type":"ContainerDied","Data":"0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118"}
Sep 11 18:08:53.102310 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.102219    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d" event={"ID":"9f70f086-8283-46f1-8af6-3a4e18367a19","Type":"ContainerDied","Data":"c73360ebd783f5d9ddf2b52fc4a6731cf2af0d73986503be405df0164fb4746f"}
Sep 11 18:08:53.102310 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.102244    2582 scope.go:117] "RemoveContainer" containerID="0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118"
Sep 11 18:08:53.112027 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.112012    2582 scope.go:117] "RemoveContainer" containerID="0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118"
Sep 11 18:08:53.112274 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:08:53.112258    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118\": container with ID starting with 0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118 not found: ID does not exist" containerID="0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118"
Sep 11 18:08:53.112326 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.112280    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118"} err="failed to get container status \"0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118\": rpc error: code = NotFound desc = could not find container \"0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118\": container with ID starting with 0e98ab3981887fa457ee02f3d7f339d24c2f9f9d13dd3452e3f705ac3c7ec118 not found: ID does not exist"
Sep 11 18:08:53.126276 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.126250    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"]
Sep 11 18:08:53.128916 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.128893    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-1bf937/rate-test-model-1bf937-kserve-7c76d64d47-br82d"]
Sep 11 18:08:53.212644 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.212611    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:08:53.747785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.747752    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6bf807a9-e755-4d4d-b61e-35f3a8c44a02" path="/var/lib/kubelet/pods/6bf807a9-e755-4d4d-b61e-35f3a8c44a02/volumes"
Sep 11 18:08:53.748194 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:53.748180    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f70f086-8283-46f1-8af6-3a4e18367a19" path="/var/lib/kubelet/pods/9f70f086-8283-46f1-8af6-3a4e18367a19/volumes"
Sep 11 18:08:56.505002 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:08:56.504961    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:04.918537 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:04.918495    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:06.560706 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:06.560670    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:07.115631 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.115596    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:07.432387 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.432273    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:07.717546 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.717453    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"]
Sep 11 18:09:07.718052 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.718032    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6bf807a9-e755-4d4d-b61e-35f3a8c44a02" containerName="main"
Sep 11 18:09:07.718139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.718054    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6bf807a9-e755-4d4d-b61e-35f3a8c44a02" containerName="main"
Sep 11 18:09:07.718139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.718065    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f70f086-8283-46f1-8af6-3a4e18367a19" containerName="main"
Sep 11 18:09:07.718139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.718074    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f70f086-8283-46f1-8af6-3a4e18367a19" containerName="main"
Sep 11 18:09:07.718587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.718195    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f70f086-8283-46f1-8af6-3a4e18367a19" containerName="main"
Sep 11 18:09:07.718587 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.718211    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6bf807a9-e755-4d4d-b61e-35f3a8c44a02" containerName="main"
Sep 11 18:09:07.721467 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.721445    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.724470 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.724450    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-f8f6a0-openshift-default-dockercfg-hv7hl\""
Sep 11 18:09:07.732200 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732180    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-data\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732304 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732235    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-token\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732304 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732265    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-podinfo\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732304 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732293    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-envoy\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732509 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732323    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istiod-ca-cert\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732509 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732365    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt42c\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-kube-api-access-kt42c\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732509 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732476    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-certs\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732657 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732530    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-socket\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.732657 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.732557    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-credential-socket\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.751254 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.751231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"]
Sep 11 18:09:07.761946 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.761919    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:07.833118 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833088    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-certs\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833298 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833132    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-socket\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833298 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833151    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-credential-socket\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833298 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-data\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833503 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-token\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833503 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833388    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-podinfo\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833503 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833413    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-envoy\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833503 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833447    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istiod-ca-cert\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833503 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833472    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt42c\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-kube-api-access-kt42c\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833774 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833548    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-credential-socket\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833774 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833624    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-certs\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833774 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833655    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-socket\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.833774 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.833713    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-data\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.834214 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.834193    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istiod-ca-cert\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.835808 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.835790    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-envoy\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.836036 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.836020    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-podinfo\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.845123 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.845102    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt42c\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-kube-api-access-kt42c\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:07.845699 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:07.845671    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-token\") pod \"e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") " pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:08.033634 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.033600    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:08.160777 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.160739    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:08.315923 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.315902    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"]
Sep 11 18:09:08.318165 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:09:08.318139    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f7d932_3e67_4ff7_af69_ca9f29d0e2bd.slice/crio-fefcc5a0629d9b6db7333458beea962738d13b21dfdc89abcbda745e5aebc526 WatchSource:0}: Error finding container fefcc5a0629d9b6db7333458beea962738d13b21dfdc89abcbda745e5aebc526: Status 404 returned error can't find the container with id fefcc5a0629d9b6db7333458beea962738d13b21dfdc89abcbda745e5aebc526
Sep 11 18:09:08.320379 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.320325    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:09:08.320472 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.320416    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:09:08.320472 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.320456    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:09:08.457366 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:08.451768    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:09.187390 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:09.187336    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" event={"ID":"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd","Type":"ContainerStarted","Data":"653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"}
Sep 11 18:09:09.187755 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:09.187438    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" event={"ID":"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd","Type":"ContainerStarted","Data":"fefcc5a0629d9b6db7333458beea962738d13b21dfdc89abcbda745e5aebc526"}
Sep 11 18:09:09.258322 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:09.258270    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podStartSLOduration=2.258254692 podStartE2EDuration="2.258254692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:09:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:09:09.256454754 +0000 UTC m=+1832.126497680" watchObservedRunningTime="2026-09-11 18:09:09.258254692 +0000 UTC m=+1832.128297707"
Sep 11 18:09:09.559915 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:09.559882    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:10.033889 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:10.033852    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:10.039045 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:10.039022    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:10.192801 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:10.192770    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:10.193892 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:10.193872    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:10.770054 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:10.770015    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:11.548190 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:11.548153    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:11.650574 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:11.650536    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:12.602230 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:12.602195    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:13.515212 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:13.515181    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:14.418022 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:14.417988    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:14.513690 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:14.513651    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:14.801669 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:14.801636    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:14.952539 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:14.952506    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:15.060173 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:15.060100    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:18.818473 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:18.818441    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:09:24.592995 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:24.592959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:29.309773 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:29.309737    2582 generic.go:356] "Generic (PLEG): container finished" podID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerID="653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2" exitCode=0
Sep 11 18:09:29.310306 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:29.309813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" event={"ID":"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd","Type":"ContainerDied","Data":"653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"}
Sep 11 18:09:29.310405 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:29.310311    2582 scope.go:117] "RemoveContainer" containerID="653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"
Sep 11 18:09:29.311447 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:29.311418    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:09:29.311563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:29.311497    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:09:29.311563 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:29.311542    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:09:30.317444 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:30.317403    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" event={"ID":"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd","Type":"ContainerStarted","Data":"978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba"}
Sep 11 18:09:30.450128 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:30.450089    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:31.033947 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:31.033911    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:31.034123 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:31.034105    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 11 18:09:31.034168 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:31.034152    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 11 18:09:32.034414 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:32.034380    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 11 18:09:32.034783 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:32.034434    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 11 18:09:33.033945 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:33.033909    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 11 18:09:33.034108 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:33.033963    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 11 18:09:34.034537 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:34.034494    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 11 18:09:34.034998 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:34.034556    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 11 18:09:35.034975 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:35.034943    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused" start-of-body=
Sep 11 18:09:35.035448 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:35.034994    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.93:15021/healthz/ready\": dial tcp 10.132.0.93:15021: connect: connection refused"
Sep 11 18:09:36.038763 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:36.038720    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:36.039169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:36.039024    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:36.039722 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:36.039703    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:09:36.796174 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:36.796126    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:38.099037 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:38.099006    2582 scope.go:117] "RemoveContainer" containerID="62c8743c8287537f81c131292ad9278898a737420b2d127c217068af900607e1"
Sep 11 18:09:38.246155 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:38.246122    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:39.085969 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:39.085936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:39.864543 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:39.864509    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:40.739610 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:40.739580    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:41.542871 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:41.542835    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:41.635250 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:41.635214    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:46.838886 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:46.838805    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v8vwn_ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844/payload-processing/0.log"
Sep 11 18:09:50.884170 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.884130    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"]
Sep 11 18:09:50.887924 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.887898    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:50.890305 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.890285    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-64600a\"/\"default-dockercfg-sbj6b\""
Sep 11 18:09:50.890907 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.890883    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-64600a\"/\"kube-root-ca.crt\""
Sep 11 18:09:50.891025 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.890953    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-64600a\"/\"openshift-service-ca.crt\""
Sep 11 18:09:50.891025 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.890993    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-64600a\"/\"ipp-route-64600a-kserve-self-signed-certs\""
Sep 11 18:09:50.899686 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.899664    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"]
Sep 11 18:09:50.916231 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.916204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:09:50.938389 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.938360    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-dshm\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:50.938499 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.938419    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-model-cache\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:50.938499 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.938446    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-home\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:50.938499 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.938466    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tls-certs\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:50.938499 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.938493    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tmp-dir\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:50.938655 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:50.938613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45rx5\" (UniqueName: \"kubernetes.io/projected/1275313b-c88e-48a3-9a99-cf4ae364ec3e-kube-api-access-45rx5\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.039742 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.039704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45rx5\" (UniqueName: \"kubernetes.io/projected/1275313b-c88e-48a3-9a99-cf4ae364ec3e-kube-api-access-45rx5\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.039919 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.039783    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-dshm\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.039919 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.039839    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-model-cache\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.039919 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.039862    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-home\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.039919 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.039889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tls-certs\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.040135 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.039935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tmp-dir\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.040515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.040486    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-model-cache\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.040896 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.040876    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-home\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.041080 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.041034    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tmp-dir\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.043057 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.042996    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-dshm\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.043917 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.043896    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tls-certs\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.048715 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.048692    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45rx5\" (UniqueName: \"kubernetes.io/projected/1275313b-c88e-48a3-9a99-cf4ae364ec3e-kube-api-access-45rx5\") pod \"ipp-route-64600a-kserve-5455c5c547-qlb4l\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") " pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.196998 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.196913    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:09:51.474165 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:51.474124    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"]
Sep 11 18:09:51.475706 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:09:51.475678    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1275313b_c88e_48a3_9a99_cf4ae364ec3e.slice/crio-3deeaf0271a6c4c29f61d8d11e174cec20927127d19ba6b7c407d101c0ac7fa1 WatchSource:0}: Error finding container 3deeaf0271a6c4c29f61d8d11e174cec20927127d19ba6b7c407d101c0ac7fa1: Status 404 returned error can't find the container with id 3deeaf0271a6c4c29f61d8d11e174cec20927127d19ba6b7c407d101c0ac7fa1
Sep 11 18:09:52.442688 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:52.442646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l" event={"ID":"1275313b-c88e-48a3-9a99-cf4ae364ec3e","Type":"ContainerStarted","Data":"1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf"}
Sep 11 18:09:52.443127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:52.442698    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l" event={"ID":"1275313b-c88e-48a3-9a99-cf4ae364ec3e","Type":"ContainerStarted","Data":"3deeaf0271a6c4c29f61d8d11e174cec20927127d19ba6b7c407d101c0ac7fa1"}
Sep 11 18:09:52.463240 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:52.463190    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l" podStartSLOduration=2.463173698 podStartE2EDuration="2.463173698s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:09:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:09:52.461463509 +0000 UTC m=+1875.331506462" watchObservedRunningTime="2026-09-11 18:09:52.463173698 +0000 UTC m=+1875.333216643"
Sep 11 18:09:55.010035 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:09:55.010001    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:10:01.197562 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:01.197474    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:10:01.197562 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:01.197539    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:10:01.209861 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:01.209831    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:10:01.508728 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:01.508696    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:10:22.002886 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:22.002851    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:10:28.398152 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:28.398121    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v8vwn_ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844/payload-processing/0.log"
Sep 11 18:10:28.819029 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:28.818988    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:29.121917 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.121842    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"]
Sep 11 18:10:29.125820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.125799    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.128743 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.128724    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-2db21a-openshift-default-dockercfg-266pj\""
Sep 11 18:10:29.138382 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.138330    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"]
Sep 11 18:10:29.164137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.164111    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:29.283659 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283625    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-data\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.283659 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283661    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-socket\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.283932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283682    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-podinfo\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.283932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283827    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-credential-socket\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.283932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283861    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pffbz\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-kube-api-access-pffbz\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.283932 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283919    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c758663-1a44-486a-b8a9-eaa30b55488b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.284150 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.283952    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-certs\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.284150 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.284012    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-envoy\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.284150 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.284034    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-token\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.383241 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.383164    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:29.385315 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385293    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-envoy\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-token\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385390    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-data\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385493 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-socket\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385553 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385529    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-podinfo\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385622 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385606    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-credential-socket\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385680 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385644    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pffbz\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-kube-api-access-pffbz\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385733 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385701    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c758663-1a44-486a-b8a9-eaa30b55488b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385806 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385783    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-socket\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385866 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385838    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-certs\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385927 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385859    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-data\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.385985 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.385966    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-credential-socket\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.386139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.386121    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-certs\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.386494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.386471    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c758663-1a44-486a-b8a9-eaa30b55488b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.387860 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.387833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-envoy\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.388097 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.388078    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-podinfo\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.393301 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.393279    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-token\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.393637 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.393619    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pffbz\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-kube-api-access-pffbz\") pod \"e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") " pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.437323 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.437302    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:29.716027 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.716002    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"]
Sep 11 18:10:29.717666 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:10:29.717639    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c758663_1a44_486a_b8a9_eaa30b55488b.slice/crio-de16c043c2c4d17e9835dc9a1969858f715b59ad8262a3fae4c0551d39218e70 WatchSource:0}: Error finding container de16c043c2c4d17e9835dc9a1969858f715b59ad8262a3fae4c0551d39218e70: Status 404 returned error can't find the container with id de16c043c2c4d17e9835dc9a1969858f715b59ad8262a3fae4c0551d39218e70
Sep 11 18:10:29.719899 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.719862    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:10:29.719991 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.719924    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:10:29.719991 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:29.719953    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:10:30.653555 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:30.653517    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" event={"ID":"6c758663-1a44-486a-b8a9-eaa30b55488b","Type":"ContainerStarted","Data":"4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"}
Sep 11 18:10:30.653985 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:30.653568    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" event={"ID":"6c758663-1a44-486a-b8a9-eaa30b55488b","Type":"ContainerStarted","Data":"de16c043c2c4d17e9835dc9a1969858f715b59ad8262a3fae4c0551d39218e70"}
Sep 11 18:10:30.677442 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:30.677394    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podStartSLOduration=1.6773792090000001 podStartE2EDuration="1.677379209s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:10:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:10:30.675045936 +0000 UTC m=+1913.545088880" watchObservedRunningTime="2026-09-11 18:10:30.677379209 +0000 UTC m=+1913.547422154"
Sep 11 18:10:31.438568 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:31.438530    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:31.444055 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:31.444028    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:31.659220 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:31.659182    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:31.660642 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:31.660618    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:31.984765 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:31.984731    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:32.712411 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:32.712380    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:32.812271 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:32.812237    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:33.066107 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:33.066065    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:34.149274 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:34.149241    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:35.100441 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:35.100402    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:35.860002 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:35.859968    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:36.683666 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:36.683629    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:36.784422 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:36.784390    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:37.014848 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:37.014810    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:37.240602 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:37.240548    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:37.334770 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:37.334687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:38.126171 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:38.126140    2582 scope.go:117] "RemoveContainer" containerID="7063dac1ee54aa8446c4dab0ab4659bc1014250a37e75584deb6b704bbba312d"
Sep 11 18:10:38.134668 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:38.134650    2582 scope.go:117] "RemoveContainer" containerID="6e013a9ce598d3494e7b9cdebb4482f043d70f5a001c6845df291e7150d786c5"
Sep 11 18:10:38.143598 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:38.143574    2582 scope.go:117] "RemoveContainer" containerID="9e0c3f0a50513d9008be85c2dfef796b27f6045ec4b09a85adf92eb890bfd49f"
Sep 11 18:10:41.011000 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:41.010959    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:10:50.776684 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:50.776651    2582 generic.go:356] "Generic (PLEG): container finished" podID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerID="4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713" exitCode=0
Sep 11 18:10:50.777047 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:50.776711    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" event={"ID":"6c758663-1a44-486a-b8a9-eaa30b55488b","Type":"ContainerDied","Data":"4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"}
Sep 11 18:10:50.777088 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:50.777059    2582 scope.go:117] "RemoveContainer" containerID="4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"
Sep 11 18:10:50.778191 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:50.778162    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:10:50.778254 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:50.778228    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:10:50.778295 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:50.778258    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 11 18:10:51.229739 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:51.229709    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:51.293093 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:51.293052    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:51.783809 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:51.783777    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" event={"ID":"6c758663-1a44-486a-b8a9-eaa30b55488b","Type":"ContainerStarted","Data":"4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704"}
Sep 11 18:10:52.437821 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:52.437785    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:52.438590 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:52.438559    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 11 18:10:52.438814 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:52.438785    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 11 18:10:53.437813 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:53.437776    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 11 18:10:53.438473 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:53.437840    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 11 18:10:54.438565 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:54.438521    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 11 18:10:54.439090 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:54.438600    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 11 18:10:55.438280 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:55.438235    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 11 18:10:55.438515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:55.438304    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 11 18:10:56.438613 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:56.438572    2582 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 11 18:10:56.439208 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:56.439171    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 11 18:10:57.442478 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:57.442448    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:57.442907 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:57.442738    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:57.443674 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:57.443656    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:10:58.785093 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:58.785050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:10:59.586165 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:10:59.586130    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:00.398100 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:00.398067    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:01.280633 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:01.280587    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:02.050239 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:02.050204    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:02.889872 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:02.889835    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:03.029766 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.029724    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:03.662320 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.662288    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"]
Sep 11 18:11:03.666999 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.666981    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:03.669568 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.669542    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:11:03.675820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.675797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"]
Sep 11 18:11:03.810818 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.810787    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e770925-cc47-4004-8123-9c337d214998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s\" (UID: \"4e770925-cc47-4004-8123-9c337d214998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:03.912104 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.912066    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e770925-cc47-4004-8123-9c337d214998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s\" (UID: \"4e770925-cc47-4004-8123-9c337d214998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:03.912832 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.912769    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e770925-cc47-4004-8123-9c337d214998-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s\" (UID: \"4e770925-cc47-4004-8123-9c337d214998\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:03.979120 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:03.979091    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:04.251216 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:04.251188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"]
Sep 11 18:11:04.252444 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:11:04.252404    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e770925_cc47_4004_8123_9c337d214998.slice/crio-c3b8e8832281157c60373528099814c76d964e69d91e77d6253a2cdcd5d4861a WatchSource:0}: Error finding container c3b8e8832281157c60373528099814c76d964e69d91e77d6253a2cdcd5d4861a: Status 404 returned error can't find the container with id c3b8e8832281157c60373528099814c76d964e69d91e77d6253a2cdcd5d4861a
Sep 11 18:11:04.734920 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:04.734881    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:04.853365 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:04.853264    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s" event={"ID":"4e770925-cc47-4004-8123-9c337d214998","Type":"ContainerStarted","Data":"aeead91dc516ddcc3cab4948f144d20743a097ded3259e20507f3124816b696a"}
Sep 11 18:11:04.853365 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:04.853324    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s" event={"ID":"4e770925-cc47-4004-8123-9c337d214998","Type":"ContainerStarted","Data":"c3b8e8832281157c60373528099814c76d964e69d91e77d6253a2cdcd5d4861a"}
Sep 11 18:11:04.876708 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:04.876651    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s" podStartSLOduration=1.876635572 podStartE2EDuration="1.876635572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:11:04.87372902 +0000 UTC m=+1947.743771964" watchObservedRunningTime="2026-09-11 18:11:04.876635572 +0000 UTC m=+1947.746678518"
Sep 11 18:11:05.551662 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:05.551628    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:05.860838 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:05.860744    2582 generic.go:356] "Generic (PLEG): container finished" podID="4e770925-cc47-4004-8123-9c337d214998" containerID="aeead91dc516ddcc3cab4948f144d20743a097ded3259e20507f3124816b696a" exitCode=0
Sep 11 18:11:05.860838 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:05.860826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s" event={"ID":"4e770925-cc47-4004-8123-9c337d214998","Type":"ContainerDied","Data":"aeead91dc516ddcc3cab4948f144d20743a097ded3259e20507f3124816b696a"}
Sep 11 18:11:06.278135 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:06.278095    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:06.999173 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:06.999145    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:07.142131 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.142039    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e770925-cc47-4004-8123-9c337d214998-maas-api-service-ca\") pod \"4e770925-cc47-4004-8123-9c337d214998\" (UID: \"4e770925-cc47-4004-8123-9c337d214998\") "
Sep 11 18:11:07.142415 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.142388    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e770925-cc47-4004-8123-9c337d214998-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4e770925-cc47-4004-8123-9c337d214998" (UID: "4e770925-cc47-4004-8123-9c337d214998"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:11:07.168048 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.168014    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:07.243309 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.243278    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4e770925-cc47-4004-8123-9c337d214998-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:07.870454 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.870417    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:07.872601 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.872577    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"
Sep 11 18:11:07.872743 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.872603    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s" event={"ID":"4e770925-cc47-4004-8123-9c337d214998","Type":"ContainerDied","Data":"c3b8e8832281157c60373528099814c76d964e69d91e77d6253a2cdcd5d4861a"}
Sep 11 18:11:07.872743 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:07.872644    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c3b8e8832281157c60373528099814c76d964e69d91e77d6253a2cdcd5d4861a"
Sep 11 18:11:08.598469 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:08.598433    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:09.272625 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:09.272592    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:09.372174 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:09.372147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:14.522927 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:14.522897    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:14.623538 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:14.623503    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:14.714938 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:14.714904    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"]
Sep 11 18:11:14.717905 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:14.717876    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-2db21a-54v6s"]
Sep 11 18:11:14.718121 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:14.718098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:14.794025 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:14.793932    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:15.122045 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:15.121949    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:15.222475 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:15.222444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:15.749115 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:15.749076    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e770925-cc47-4004-8123-9c337d214998" path="/var/lib/kubelet/pods/4e770925-cc47-4004-8123-9c337d214998/volumes"
Sep 11 18:11:16.068949 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:16.068848    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:16.830229 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:16.830186    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:17.991442 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:17.991405    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:18.002031 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:18.002000    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:11:18.857800 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:18.857765    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:21.338340 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:21.338296    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:21.409240 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:21.409208    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:21.603161 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:21.603075    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"]
Sep 11 18:11:21.603359 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:21.603324    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy" containerID="cri-o://4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704" gracePeriod=30
Sep 11 18:11:21.615825 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:21.615796    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:25.457595 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.457560    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"]
Sep 11 18:11:25.458014 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.457999    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e770925-cc47-4004-8123-9c337d214998" containerName="revoke-keys"
Sep 11 18:11:25.458061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.458015    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e770925-cc47-4004-8123-9c337d214998" containerName="revoke-keys"
Sep 11 18:11:25.458103 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.458094    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e770925-cc47-4004-8123-9c337d214998" containerName="revoke-keys"
Sep 11 18:11:25.461466 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.461446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:25.463788 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.463769    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:11:25.469667 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.469646    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"]
Sep 11 18:11:25.511614 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.511586    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be172b9-f5d6-4bb7-90d0-638d82ab12ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx\" (UID: \"5be172b9-f5d6-4bb7-90d0-638d82ab12ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:25.613151 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.613115    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be172b9-f5d6-4bb7-90d0-638d82ab12ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx\" (UID: \"5be172b9-f5d6-4bb7-90d0-638d82ab12ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:25.613772 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.613753    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be172b9-f5d6-4bb7-90d0-638d82ab12ad-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx\" (UID: \"5be172b9-f5d6-4bb7-90d0-638d82ab12ad\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:25.772855 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.772818    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:25.782408 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:25.782376    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:26.060921 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.060894    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"]
Sep 11 18:11:26.062614 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:11:26.062588    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5be172b9_f5d6_4bb7_90d0_638d82ab12ad.slice/crio-5ff29e4409a06a0c2a97fb6053c7188d6c5edafede3897811cf23949af6ba1c1 WatchSource:0}: Error finding container 5ff29e4409a06a0c2a97fb6053c7188d6c5edafede3897811cf23949af6ba1c1: Status 404 returned error can't find the container with id 5ff29e4409a06a0c2a97fb6053c7188d6c5edafede3897811cf23949af6ba1c1
Sep 11 18:11:26.777652 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.777620    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:26.860442 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.860407    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:11:26.926864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pffbz\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-kube-api-access-pffbz\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.926864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926827    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-data\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.926864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926846    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-credential-socket\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.926864 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926861    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c758663-1a44-486a-b8a9-eaa30b55488b-istiod-ca-cert\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.927199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926881    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-socket\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.927199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926910    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-token\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.927199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926936    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-podinfo\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.927199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.926987    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-certs\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.927199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.927018    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-envoy\") pod \"6c758663-1a44-486a-b8a9-eaa30b55488b\" (UID: \"6c758663-1a44-486a-b8a9-eaa30b55488b\") "
Sep 11 18:11:26.927749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.927226    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:11:26.927749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.927273    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-data" (OuterVolumeSpecName: "istio-data") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:11:26.927749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.927281    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c758663-1a44-486a-b8a9-eaa30b55488b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:11:26.927749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.927472    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:11:26.927749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.927492    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:11:26.929130 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.929108    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-kube-api-access-pffbz" (OuterVolumeSpecName: "kube-api-access-pffbz") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "kube-api-access-pffbz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:11:26.929269 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.929251    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:11:26.929693 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.929671    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:11:26.929749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.929686    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6c758663-1a44-486a-b8a9-eaa30b55488b" (UID: "6c758663-1a44-486a-b8a9-eaa30b55488b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:11:26.975826 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.975791    2582 generic.go:356] "Generic (PLEG): container finished" podID="5be172b9-f5d6-4bb7-90d0-638d82ab12ad" containerID="1adeb14786a9b5b815d0274bfc0c60940ac0fb655c452e5df826d8717530ba8c" exitCode=0
Sep 11 18:11:26.975968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.975844    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx" event={"ID":"5be172b9-f5d6-4bb7-90d0-638d82ab12ad","Type":"ContainerDied","Data":"1adeb14786a9b5b815d0274bfc0c60940ac0fb655c452e5df826d8717530ba8c"}
Sep 11 18:11:26.975968 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.975909    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx" event={"ID":"5be172b9-f5d6-4bb7-90d0-638d82ab12ad","Type":"ContainerStarted","Data":"5ff29e4409a06a0c2a97fb6053c7188d6c5edafede3897811cf23949af6ba1c1"}
Sep 11 18:11:26.977600 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.977579    2582 generic.go:356] "Generic (PLEG): container finished" podID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerID="4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704" exitCode=0
Sep 11 18:11:26.977694 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.977605    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" event={"ID":"6c758663-1a44-486a-b8a9-eaa30b55488b","Type":"ContainerDied","Data":"4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704"}
Sep 11 18:11:26.977694 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.977621    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"
Sep 11 18:11:26.977694 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.977653    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz" event={"ID":"6c758663-1a44-486a-b8a9-eaa30b55488b","Type":"ContainerDied","Data":"de16c043c2c4d17e9835dc9a1969858f715b59ad8262a3fae4c0551d39218e70"}
Sep 11 18:11:26.977694 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.977675    2582 scope.go:117] "RemoveContainer" containerID="4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704"
Sep 11 18:11:26.987737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.987717    2582 scope.go:117] "RemoveContainer" containerID="4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"
Sep 11 18:11:26.996137 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.996119    2582 scope.go:117] "RemoveContainer" containerID="4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704"
Sep 11 18:11:26.996516 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:11:26.996481    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704\": container with ID starting with 4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704 not found: ID does not exist" containerID="4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704"
Sep 11 18:11:26.996600 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.996520    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704"} err="failed to get container status \"4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704\": rpc error: code = NotFound desc = could not find container \"4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704\": container with ID starting with 4d6dccbdcb7fdb2dadbcdf2c42737ab0c62749eb0bcc4d2de3fbd52cdb14e704 not found: ID does not exist"
Sep 11 18:11:26.996600 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.996543    2582 scope.go:117] "RemoveContainer" containerID="4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"
Sep 11 18:11:26.997083 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:11:26.997061    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713\": container with ID starting with 4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713 not found: ID does not exist" containerID="4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"
Sep 11 18:11:26.997131 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:26.997093    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713"} err="failed to get container status \"4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713\": rpc error: code = NotFound desc = could not find container \"4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713\": container with ID starting with 4bc61c6658778b22609fa15997c6cd0af4234649c5c9ac440562356560867713 not found: ID does not exist"
Sep 11 18:11:27.010957 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.010935    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"]
Sep 11 18:11:27.015266 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.015245    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-2db21a-openshift-default-5b7fd6547b-psfdz"]
Sep 11 18:11:27.028716 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028694    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028716 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028715    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028725    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028733    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pffbz\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-kube-api-access-pffbz\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028742    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028750    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028758    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c758663-1a44-486a-b8a9-eaa30b55488b-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028765    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c758663-1a44-486a-b8a9-eaa30b55488b-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.028828 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.028773    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c758663-1a44-486a-b8a9-eaa30b55488b-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:27.472430 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.472400    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:27.749000 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:27.748969    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" path="/var/lib/kubelet/pods/6c758663-1a44-486a-b8a9-eaa30b55488b/volumes"
Sep 11 18:11:28.121536 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.121510    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:28.171632 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.171589    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:28.239896 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.239860    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be172b9-f5d6-4bb7-90d0-638d82ab12ad-maas-api-service-ca\") pod \"5be172b9-f5d6-4bb7-90d0-638d82ab12ad\" (UID: \"5be172b9-f5d6-4bb7-90d0-638d82ab12ad\") "
Sep 11 18:11:28.240253 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.240230    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5be172b9-f5d6-4bb7-90d0-638d82ab12ad-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5be172b9-f5d6-4bb7-90d0-638d82ab12ad" (UID: "5be172b9-f5d6-4bb7-90d0-638d82ab12ad"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:11:28.340934 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.340859    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5be172b9-f5d6-4bb7-90d0-638d82ab12ad-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:11:28.991058 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.991023    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx" event={"ID":"5be172b9-f5d6-4bb7-90d0-638d82ab12ad","Type":"ContainerDied","Data":"5ff29e4409a06a0c2a97fb6053c7188d6c5edafede3897811cf23949af6ba1c1"}
Sep 11 18:11:28.991314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.991070    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5ff29e4409a06a0c2a97fb6053c7188d6c5edafede3897811cf23949af6ba1c1"
Sep 11 18:11:28.991314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:28.991033    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"
Sep 11 18:11:29.024223 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:29.024188    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:29.737229 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:29.737161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:38.169838 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:38.169806    2582 scope.go:117] "RemoveContainer" containerID="8a2114dd438b798d13a0f4b8f1fae38ba34e1cf738734c8557430adb78d4dae1"
Sep 11 18:11:38.178459 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:38.178437    2582 scope.go:117] "RemoveContainer" containerID="5b5156c1af2d71a9606aa703e78b7b5ec3a662f2073382876b65efa31a4eb287"
Sep 11 18:11:38.607779 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:38.607744    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:11:41.160388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.160332    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.234153 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.234124    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.310806 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.310771    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.357161 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.357130    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"]
Sep 11 18:11:41.362572 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.362547    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-64600a-r5bsx"]
Sep 11 18:11:41.383547 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.383522    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.750303 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.750261    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5be172b9-f5d6-4bb7-90d0-638d82ab12ad" path="/var/lib/kubelet/pods/5be172b9-f5d6-4bb7-90d0-638d82ab12ad/volumes"
Sep 11 18:11:41.771476 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.771437    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:41.874764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:41.874716    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:42.613777 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:42.613743    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:43.589125 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:43.589098    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:44.375376 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:44.375313    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:45.179874 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:45.179840    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:46.770852 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:46.770816    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"]
Sep 11 18:11:46.866629 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:46.866598    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:11:51.702502 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:11:51.702469    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:12:01.789861 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:01.789823    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l" podUID="1275313b-c88e-48a3-9a99-cf4ae364ec3e" containerName="main" containerID="cri-o://1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf" gracePeriod=45
Sep 11 18:12:02.038901 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.038875    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:12:02.039815 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.039795    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-model-cache\") pod \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") "
Sep 11 18:12:02.039970 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.039826    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-dshm\") pod \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") "
Sep 11 18:12:02.039970 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.039877    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45rx5\" (UniqueName: \"kubernetes.io/projected/1275313b-c88e-48a3-9a99-cf4ae364ec3e-kube-api-access-45rx5\") pod \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") "
Sep 11 18:12:02.039970 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.039913    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-home\") pod \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") "
Sep 11 18:12:02.039970 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.039943    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tmp-dir\") pod \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") "
Sep 11 18:12:02.040210 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040038    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-model-cache" (OuterVolumeSpecName: "model-cache") pod "1275313b-c88e-48a3-9a99-cf4ae364ec3e" (UID: "1275313b-c88e-48a3-9a99-cf4ae364ec3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:02.040210 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040060    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tls-certs\") pod \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\" (UID: \"1275313b-c88e-48a3-9a99-cf4ae364ec3e\") "
Sep 11 18:12:02.040314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040250    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-home" (OuterVolumeSpecName: "home") pod "1275313b-c88e-48a3-9a99-cf4ae364ec3e" (UID: "1275313b-c88e-48a3-9a99-cf4ae364ec3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:02.040314 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040275    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1275313b-c88e-48a3-9a99-cf4ae364ec3e" (UID: "1275313b-c88e-48a3-9a99-cf4ae364ec3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:02.040467 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040403    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:02.040467 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040415    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:02.040467 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.040424    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:02.042187 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.042166    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-dshm" (OuterVolumeSpecName: "dshm") pod "1275313b-c88e-48a3-9a99-cf4ae364ec3e" (UID: "1275313b-c88e-48a3-9a99-cf4ae364ec3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:02.042267 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.042182    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1275313b-c88e-48a3-9a99-cf4ae364ec3e-kube-api-access-45rx5" (OuterVolumeSpecName: "kube-api-access-45rx5") pod "1275313b-c88e-48a3-9a99-cf4ae364ec3e" (UID: "1275313b-c88e-48a3-9a99-cf4ae364ec3e"). InnerVolumeSpecName "kube-api-access-45rx5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:02.042267 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.042203    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1275313b-c88e-48a3-9a99-cf4ae364ec3e" (UID: "1275313b-c88e-48a3-9a99-cf4ae364ec3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:12:02.141139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.141106    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45rx5\" (UniqueName: \"kubernetes.io/projected/1275313b-c88e-48a3-9a99-cf4ae364ec3e-kube-api-access-45rx5\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:02.141139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.141133    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1275313b-c88e-48a3-9a99-cf4ae364ec3e-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:02.141139 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.141141    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1275313b-c88e-48a3-9a99-cf4ae364ec3e-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:02.166457 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.166420    2582 generic.go:356] "Generic (PLEG): container finished" podID="1275313b-c88e-48a3-9a99-cf4ae364ec3e" containerID="1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf" exitCode=0
Sep 11 18:12:02.166611 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.166470    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"
Sep 11 18:12:02.166611 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.166506    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l" event={"ID":"1275313b-c88e-48a3-9a99-cf4ae364ec3e","Type":"ContainerDied","Data":"1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf"}
Sep 11 18:12:02.166611 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.166549    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l" event={"ID":"1275313b-c88e-48a3-9a99-cf4ae364ec3e","Type":"ContainerDied","Data":"3deeaf0271a6c4c29f61d8d11e174cec20927127d19ba6b7c407d101c0ac7fa1"}
Sep 11 18:12:02.166611 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.166567    2582 scope.go:117] "RemoveContainer" containerID="1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf"
Sep 11 18:12:02.176751 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.176732    2582 scope.go:117] "RemoveContainer" containerID="1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf"
Sep 11 18:12:02.177010 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:12:02.176987    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf\": container with ID starting with 1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf not found: ID does not exist" containerID="1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf"
Sep 11 18:12:02.177105 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.177015    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf"} err="failed to get container status \"1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf\": rpc error: code = NotFound desc = could not find container \"1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf\": container with ID starting with 1588c6a5b5df7824cef8dea649308873c38b01cae7239acc5c580bdf6eae5dcf not found: ID does not exist"
Sep 11 18:12:02.189735 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.189709    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"]
Sep 11 18:12:02.195428 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:02.195406    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-64600a/ipp-route-64600a-kserve-5455c5c547-qlb4l"]
Sep 11 18:12:03.746760 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:03.746732    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1275313b-c88e-48a3-9a99-cf4ae364ec3e" path="/var/lib/kubelet/pods/1275313b-c88e-48a3-9a99-cf4ae364ec3e/volumes"
Sep 11 18:12:23.626736 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:23.626699    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:24.194461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:24.194422    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:29.928005 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:29.927959    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:30.048309 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.048275    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:30.220967 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.220889    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:30.592713 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.592679    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"]
Sep 11 18:12:30.593073 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593058    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5be172b9-f5d6-4bb7-90d0-638d82ab12ad" containerName="revoke-keys"
Sep 11 18:12:30.593127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593074    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="5be172b9-f5d6-4bb7-90d0-638d82ab12ad" containerName="revoke-keys"
Sep 11 18:12:30.593127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593090    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1275313b-c88e-48a3-9a99-cf4ae364ec3e" containerName="main"
Sep 11 18:12:30.593127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593097    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="1275313b-c88e-48a3-9a99-cf4ae364ec3e" containerName="main"
Sep 11 18:12:30.593127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593103    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy"
Sep 11 18:12:30.593127 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593110    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy"
Sep 11 18:12:30.593286 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593180    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="1275313b-c88e-48a3-9a99-cf4ae364ec3e" containerName="main"
Sep 11 18:12:30.593286 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593191    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy"
Sep 11 18:12:30.593286 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593200    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy"
Sep 11 18:12:30.593286 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.593205    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="5be172b9-f5d6-4bb7-90d0-638d82ab12ad" containerName="revoke-keys"
Sep 11 18:12:30.596323 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.596306    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:30.598685 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.598659    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:12:30.604006 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.603981    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"]
Sep 11 18:12:30.687047 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.687015    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b194411e-62aa-48ac-86ae-45eb6aa077b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv\" (UID: \"b194411e-62aa-48ac-86ae-45eb6aa077b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:30.788160 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.788123    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b194411e-62aa-48ac-86ae-45eb6aa077b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv\" (UID: \"b194411e-62aa-48ac-86ae-45eb6aa077b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:30.788774 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.788754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b194411e-62aa-48ac-86ae-45eb6aa077b9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv\" (UID: \"b194411e-62aa-48ac-86ae-45eb6aa077b9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:30.908401 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.908330    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:30.918910 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:30.918872    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:31.183874 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:31.183849    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"]
Sep 11 18:12:31.185919 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:12:31.185889    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb194411e_62aa_48ac_86ae_45eb6aa077b9.slice/crio-f563f8f6b2e14f09a211a65fdc88e7975e2b8e255435ec1121c5ea4e25fcd748 WatchSource:0}: Error finding container f563f8f6b2e14f09a211a65fdc88e7975e2b8e255435ec1121c5ea4e25fcd748: Status 404 returned error can't find the container with id f563f8f6b2e14f09a211a65fdc88e7975e2b8e255435ec1121c5ea4e25fcd748
Sep 11 18:12:31.318559 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:31.318529    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv" event={"ID":"b194411e-62aa-48ac-86ae-45eb6aa077b9","Type":"ContainerStarted","Data":"f563f8f6b2e14f09a211a65fdc88e7975e2b8e255435ec1121c5ea4e25fcd748"}
Sep 11 18:12:31.692105 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:31.692068    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:32.325652 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:32.325613    2582 generic.go:356] "Generic (PLEG): container finished" podID="b194411e-62aa-48ac-86ae-45eb6aa077b9" containerID="ba646a895afe72f266e1491ab878bff7c0d0f719c6265a29b3d06658c98fe4c8" exitCode=0
Sep 11 18:12:32.326068 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:32.325696    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv" event={"ID":"b194411e-62aa-48ac-86ae-45eb6aa077b9","Type":"ContainerDied","Data":"ba646a895afe72f266e1491ab878bff7c0d0f719c6265a29b3d06658c98fe4c8"}
Sep 11 18:12:32.451996 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:32.451962    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:33.383107 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:33.383010    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:33.470652 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:33.470630    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:33.615924 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:33.615830    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b194411e-62aa-48ac-86ae-45eb6aa077b9-maas-api-service-ca\") pod \"b194411e-62aa-48ac-86ae-45eb6aa077b9\" (UID: \"b194411e-62aa-48ac-86ae-45eb6aa077b9\") "
Sep 11 18:12:33.616216 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:33.616191    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b194411e-62aa-48ac-86ae-45eb6aa077b9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b194411e-62aa-48ac-86ae-45eb6aa077b9" (UID: "b194411e-62aa-48ac-86ae-45eb6aa077b9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:33.717419 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:33.717382    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b194411e-62aa-48ac-86ae-45eb6aa077b9-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:34.125271 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:34.125238    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:34.338404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:34.338322    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"
Sep 11 18:12:34.338404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:34.338369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv" event={"ID":"b194411e-62aa-48ac-86ae-45eb6aa077b9","Type":"ContainerDied","Data":"f563f8f6b2e14f09a211a65fdc88e7975e2b8e255435ec1121c5ea4e25fcd748"}
Sep 11 18:12:34.338617 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:34.338427    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f563f8f6b2e14f09a211a65fdc88e7975e2b8e255435ec1121c5ea4e25fcd748"
Sep 11 18:12:41.208135 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.208060    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.301745 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.301701    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.398553 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.398513    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.415253 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.415223    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"]
Sep 11 18:12:41.418229 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.418204    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-f8f6a0-m57hv"]
Sep 11 18:12:41.466140 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.466041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.747633 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.747586    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b194411e-62aa-48ac-86ae-45eb6aa077b9" path="/var/lib/kubelet/pods/b194411e-62aa-48ac-86ae-45eb6aa077b9/volumes"
Sep 11 18:12:41.842664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.842630    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:41.916899 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:41.916862    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:42.606962 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:42.606927    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:43.271879 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:43.271843    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:44.500426 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:44.500392    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:12:45.130089 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:45.130050    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:48.359947 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:48.359913    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:48.462773 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:48.462720    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:48.659944 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:48.659857    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"]
Sep 11 18:12:48.660126 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:48.660099    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" containerID="cri-o://978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba" gracePeriod=30
Sep 11 18:12:48.686147 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:48.686108    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:49.070664 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.070634    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"]
Sep 11 18:12:49.071024 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.071011    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b194411e-62aa-48ac-86ae-45eb6aa077b9" containerName="revoke-keys"
Sep 11 18:12:49.071072 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.071025    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b194411e-62aa-48ac-86ae-45eb6aa077b9" containerName="revoke-keys"
Sep 11 18:12:49.071072 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.071048    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy"
Sep 11 18:12:49.071072 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.071054    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c758663-1a44-486a-b8a9-eaa30b55488b" containerName="istio-proxy"
Sep 11 18:12:49.071166 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.071112    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b194411e-62aa-48ac-86ae-45eb6aa077b9" containerName="revoke-keys"
Sep 11 18:12:49.074399 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.074381    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:49.077714 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.077693    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:12:49.100764 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.100740    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"]
Sep 11 18:12:49.257736 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.257704    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5eb5dd-883f-4d6d-a8cf-41bff8076865-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx\" (UID: \"3c5eb5dd-883f-4d6d-a8cf-41bff8076865\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:49.337228 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.337148    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:49.358362 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.358323    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5eb5dd-883f-4d6d-a8cf-41bff8076865-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx\" (UID: \"3c5eb5dd-883f-4d6d-a8cf-41bff8076865\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:49.358976 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.358958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5eb5dd-883f-4d6d-a8cf-41bff8076865-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx\" (UID: \"3c5eb5dd-883f-4d6d-a8cf-41bff8076865\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:49.384323 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.384297    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:49.665979 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:49.665905    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"]
Sep 11 18:12:49.669617 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:12:49.669591    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c5eb5dd_883f_4d6d_a8cf_41bff8076865.slice/crio-68b409d5dc15c47ff087ee5e5c6e01b05b17276d53f29edd72124dd732e37f0d WatchSource:0}: Error finding container 68b409d5dc15c47ff087ee5e5c6e01b05b17276d53f29edd72124dd732e37f0d: Status 404 returned error can't find the container with id 68b409d5dc15c47ff087ee5e5c6e01b05b17276d53f29edd72124dd732e37f0d
Sep 11 18:12:50.430465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:50.430430    2582 generic.go:356] "Generic (PLEG): container finished" podID="3c5eb5dd-883f-4d6d-a8cf-41bff8076865" containerID="977f7e1b5868d7b3a2a7f3092c5bca9b5c4022d376df568189feca92f44b03e0" exitCode=0
Sep 11 18:12:50.430848 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:50.430523    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx" event={"ID":"3c5eb5dd-883f-4d6d-a8cf-41bff8076865","Type":"ContainerDied","Data":"977f7e1b5868d7b3a2a7f3092c5bca9b5c4022d376df568189feca92f44b03e0"}
Sep 11 18:12:50.430848 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:50.430587    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx" event={"ID":"3c5eb5dd-883f-4d6d-a8cf-41bff8076865","Type":"ContainerStarted","Data":"68b409d5dc15c47ff087ee5e5c6e01b05b17276d53f29edd72124dd732e37f0d"}
Sep 11 18:12:50.546119 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:50.546073    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:51.041852 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.041806    2582 patch_prober.go:28] interesting pod/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 18:12:51.042055 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.041878    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 18:12:51.369045 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.368951    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:51.572647 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.572612    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:51.679773 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.679679    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5eb5dd-883f-4d6d-a8cf-41bff8076865-maas-api-service-ca\") pod \"3c5eb5dd-883f-4d6d-a8cf-41bff8076865\" (UID: \"3c5eb5dd-883f-4d6d-a8cf-41bff8076865\") "
Sep 11 18:12:51.680082 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.680055    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c5eb5dd-883f-4d6d-a8cf-41bff8076865-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3c5eb5dd-883f-4d6d-a8cf-41bff8076865" (UID: "3c5eb5dd-883f-4d6d-a8cf-41bff8076865"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:51.781132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:51.781099    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c5eb5dd-883f-4d6d-a8cf-41bff8076865-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:52.180181 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:52.180147    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:52.443715 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:52.443639    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"
Sep 11 18:12:52.443855 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:52.443634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx" event={"ID":"3c5eb5dd-883f-4d6d-a8cf-41bff8076865","Type":"ContainerDied","Data":"68b409d5dc15c47ff087ee5e5c6e01b05b17276d53f29edd72124dd732e37f0d"}
Sep 11 18:12:52.443855 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:52.443767    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68b409d5dc15c47ff087ee5e5c6e01b05b17276d53f29edd72124dd732e37f0d"
Sep 11 18:12:53.918155 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:53.918132    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:12:54.100572 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100540    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-token\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.100572 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100575    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-socket\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.100785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100611    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istiod-ca-cert\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.100785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100722    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-data\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.100898 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100788    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-credential-socket\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.100898 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100817    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kt42c\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-kube-api-access-kt42c\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.100898 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100820    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:54.101063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100906    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-podinfo\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.101063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100934    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-envoy\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.101063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100959    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-certs\") pod \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\" (UID: \"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd\") "
Sep 11 18:12:54.101063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.100984    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:12:54.101063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101012    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:54.101063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:54.101388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101266    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.101388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101273    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:54.101388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101284    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.101388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101300    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.101388 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.101315    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.102848 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.102823    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:54.103098 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.103078    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:12:54.103168 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.103081    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:12:54.103168 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.103098    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-kube-api-access-kt42c" (OuterVolumeSpecName: "kube-api-access-kt42c") pod "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" (UID: "d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd"). InnerVolumeSpecName "kube-api-access-kt42c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:12:54.202461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.202421    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kt42c\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-kube-api-access-kt42c\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.202461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.202464    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.202633 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.202477    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.202633 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.202485    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.202633 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.202493    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:12:54.456111 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.456023    2582 generic.go:356] "Generic (PLEG): container finished" podID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerID="978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba" exitCode=0
Sep 11 18:12:54.456111 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.456062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" event={"ID":"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd","Type":"ContainerDied","Data":"978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba"}
Sep 11 18:12:54.456111 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.456076    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"
Sep 11 18:12:54.456111 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.456099    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc" event={"ID":"d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd","Type":"ContainerDied","Data":"fefcc5a0629d9b6db7333458beea962738d13b21dfdc89abcbda745e5aebc526"}
Sep 11 18:12:54.456486 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.456120    2582 scope.go:117] "RemoveContainer" containerID="978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba"
Sep 11 18:12:54.466031 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.466012    2582 scope.go:117] "RemoveContainer" containerID="653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"
Sep 11 18:12:54.474020 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.473997    2582 scope.go:117] "RemoveContainer" containerID="978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba"
Sep 11 18:12:54.474328 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:12:54.474306    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba\": container with ID starting with 978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba not found: ID does not exist" containerID="978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba"
Sep 11 18:12:54.474409 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.474337    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba"} err="failed to get container status \"978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba\": rpc error: code = NotFound desc = could not find container \"978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba\": container with ID starting with 978734f8daa7888c405104198a3f8e012cefabf7c572de3bd0bacde86c24c8ba not found: ID does not exist"
Sep 11 18:12:54.474409 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.474375    2582 scope.go:117] "RemoveContainer" containerID="653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"
Sep 11 18:12:54.474617 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:12:54.474594    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2\": container with ID starting with 653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2 not found: ID does not exist" containerID="653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"
Sep 11 18:12:54.474665 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.474622    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2"} err="failed to get container status \"653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2\": rpc error: code = NotFound desc = could not find container \"653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2\": container with ID starting with 653a3fb4048db456ca44cb6b369154e9f2c762fd4bf087f3b4d1f83498eb46b2 not found: ID does not exist"
Sep 11 18:12:54.478915 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.478892    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"]
Sep 11 18:12:54.482132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:54.482111    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-f8f6a0-openshift-default-6bdc99cfb6-256sc"]
Sep 11 18:12:55.746410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:55.746373    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" path="/var/lib/kubelet/pods/d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd/volumes"
Sep 11 18:12:59.556908 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.556857    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:59.631959 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.631929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:59.699279 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.699249    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:59.770178 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.770145    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"]
Sep 11 18:12:59.772725 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.772700    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-a18ac6-gcqqx"]
Sep 11 18:12:59.774546 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.774516    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:12:59.853190 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:12:59.853102    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:00.167937 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:00.167837    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:00.272666 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:00.272635    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:00.913724 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:00.913687    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:01.746294 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:01.746253    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c5eb5dd-883f-4d6d-a8cf-41bff8076865" path="/var/lib/kubelet/pods/3c5eb5dd-883f-4d6d-a8cf-41bff8076865/volumes"
Sep 11 18:13:02.328674 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:02.328637    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:13:06.496333 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:06.496294    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:06.577335 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:06.577289    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:06.793687 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:06.793600    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:07.239643 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.239614    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"]
Sep 11 18:13:07.240077 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240058    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy"
Sep 11 18:13:07.240169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240079    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy"
Sep 11 18:13:07.240169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240104    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy"
Sep 11 18:13:07.240169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240113    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy"
Sep 11 18:13:07.240169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240145    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c5eb5dd-883f-4d6d-a8cf-41bff8076865" containerName="revoke-keys"
Sep 11 18:13:07.240169 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240153    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c5eb5dd-883f-4d6d-a8cf-41bff8076865" containerName="revoke-keys"
Sep 11 18:13:07.240471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240245    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy"
Sep 11 18:13:07.240471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240261    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f7d932-3e67-4ff7-af69-ca9f29d0e2bd" containerName="istio-proxy"
Sep 11 18:13:07.240471 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.240271    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c5eb5dd-883f-4d6d-a8cf-41bff8076865" containerName="revoke-keys"
Sep 11 18:13:07.245069 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.245049    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:07.249073 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.249026    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:13:07.263614 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.263592    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"]
Sep 11 18:13:07.313441 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.313411    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56c4f1fe-98e6-4d26-82ea-76ef081efbb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r\" (UID: \"56c4f1fe-98e6-4d26-82ea-76ef081efbb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:07.367706 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.367677    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:07.414154 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.414126    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56c4f1fe-98e6-4d26-82ea-76ef081efbb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r\" (UID: \"56c4f1fe-98e6-4d26-82ea-76ef081efbb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:07.414851 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.414826    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56c4f1fe-98e6-4d26-82ea-76ef081efbb9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r\" (UID: \"56c4f1fe-98e6-4d26-82ea-76ef081efbb9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:07.555457 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.555338    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:07.826908 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:07.826876    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"]
Sep 11 18:13:07.829178 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:13:07.829146    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c4f1fe_98e6_4d26_82ea_76ef081efbb9.slice/crio-b6121be4567c2faecb94beaa819f3a537929b4d89ed7b12545b1e0d7a6bfa684 WatchSource:0}: Error finding container b6121be4567c2faecb94beaa819f3a537929b4d89ed7b12545b1e0d7a6bfa684: Status 404 returned error can't find the container with id b6121be4567c2faecb94beaa819f3a537929b4d89ed7b12545b1e0d7a6bfa684
Sep 11 18:13:08.174201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:08.174108    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:08.530554 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:08.530514    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r" event={"ID":"56c4f1fe-98e6-4d26-82ea-76ef081efbb9","Type":"ContainerStarted","Data":"a7e69948c427be99c209b9a3686ab64758d5c11e30885b529e6e1884beec118e"}
Sep 11 18:13:08.530768 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:08.530614    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r" event={"ID":"56c4f1fe-98e6-4d26-82ea-76ef081efbb9","Type":"ContainerStarted","Data":"b6121be4567c2faecb94beaa819f3a537929b4d89ed7b12545b1e0d7a6bfa684"}
Sep 11 18:13:08.547054 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:08.547003    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r" podStartSLOduration=1.546989355 podStartE2EDuration="1.546989355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:13:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:13:08.545853016 +0000 UTC m=+2071.415895963" watchObservedRunningTime="2026-09-11 18:13:08.546989355 +0000 UTC m=+2071.417032300"
Sep 11 18:13:08.975610 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:08.975523    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:09.536830 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:09.536793    2582 generic.go:356] "Generic (PLEG): container finished" podID="56c4f1fe-98e6-4d26-82ea-76ef081efbb9" containerID="a7e69948c427be99c209b9a3686ab64758d5c11e30885b529e6e1884beec118e" exitCode=0
Sep 11 18:13:09.537034 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:09.536872    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r" event={"ID":"56c4f1fe-98e6-4d26-82ea-76ef081efbb9","Type":"ContainerDied","Data":"a7e69948c427be99c209b9a3686ab64758d5c11e30885b529e6e1884beec118e"}
Sep 11 18:13:10.675410 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:10.675385    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:10.742750 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:10.742711    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56c4f1fe-98e6-4d26-82ea-76ef081efbb9-maas-api-service-ca\") pod \"56c4f1fe-98e6-4d26-82ea-76ef081efbb9\" (UID: \"56c4f1fe-98e6-4d26-82ea-76ef081efbb9\") "
Sep 11 18:13:10.743098 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:10.743073    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56c4f1fe-98e6-4d26-82ea-76ef081efbb9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "56c4f1fe-98e6-4d26-82ea-76ef081efbb9" (UID: "56c4f1fe-98e6-4d26-82ea-76ef081efbb9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:10.843583 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:10.843493    2582 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/56c4f1fe-98e6-4d26-82ea-76ef081efbb9-maas-api-service-ca\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:11.549307 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:11.549275    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"
Sep 11 18:13:11.549505 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:11.549310    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r" event={"ID":"56c4f1fe-98e6-4d26-82ea-76ef081efbb9","Type":"ContainerDied","Data":"b6121be4567c2faecb94beaa819f3a537929b4d89ed7b12545b1e0d7a6bfa684"}
Sep 11 18:13:11.549505 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:11.549387    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6121be4567c2faecb94beaa819f3a537929b4d89ed7b12545b1e0d7a6bfa684"
Sep 11 18:13:17.726473 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:17.726443    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"]
Sep 11 18:13:17.732064 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:17.732037    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-1bf937-29819160-l4mwn"]
Sep 11 18:13:17.740192 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:17.740161    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:17.748091 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:17.748064    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ebb2753-613c-41d7-b50c-dcc7e6705451" path="/var/lib/kubelet/pods/7ebb2753-613c-41d7-b50c-dcc7e6705451/volumes"
Sep 11 18:13:17.826991 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:17.826960    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:17.930548 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:17.930514    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:18.018640 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:18.018610    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"]
Sep 11 18:13:18.022785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:18.022760    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w3-1bf937-pdk4r"]
Sep 11 18:13:18.024463 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:18.024442    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:18.101957 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:18.101929    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:18.522972 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:18.522944    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:18.637689 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:18.637650    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:19.748343 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:19.748305    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56c4f1fe-98e6-4d26-82ea-76ef081efbb9" path="/var/lib/kubelet/pods/56c4f1fe-98e6-4d26-82ea-76ef081efbb9/volumes"
Sep 11 18:13:20.498612 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:20.498565    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:13:24.534143 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:24.534104    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:24.633996 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:24.633963    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:24.782914 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:24.782880    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"]
Sep 11 18:13:24.783116 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:24.783093    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy" containerID="cri-o://4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67" gracePeriod=30
Sep 11 18:13:24.809537 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:24.809455    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:13:29.305652 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:29.305620    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:13:30.030084 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.030057    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 18:13:30.110124 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110037    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-znbzr\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-kube-api-access-znbzr\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110124 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110097    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istiod-ca-cert\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110124 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110117    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-socket\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110146    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-token\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110180    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-credential-socket\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110205    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-envoy\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110380    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-data\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110453 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110429    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-certs\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110458    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-podinfo\") pod \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\" (UID: \"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58\") "
Sep 11 18:13:30.110711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110493    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:30.110711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110513    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:30.110711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110581    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:30.110711 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110637    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 18:13:30.110974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110730    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:30.110974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110865    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-data\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.110974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110878    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.110974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110888    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istiod-ca-cert\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.110974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110896    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-workload-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.110974 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.110905    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-credential-socket\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.112425 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.112391    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:13:30.112541 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.112480    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:13:30.112621 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.112603    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-kube-api-access-znbzr" (OuterVolumeSpecName: "kube-api-access-znbzr") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "kube-api-access-znbzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:13:30.112698 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.112675    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" (UID: "8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 18:13:30.212265 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.212229    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-podinfo\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.212265 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.212259    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-znbzr\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-kube-api-access-znbzr\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.212499 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.212275    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-token\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.212499 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.212288    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58-istio-envoy\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:13:30.658612 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.658573    2582 generic.go:356] "Generic (PLEG): container finished" podID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerID="4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67" exitCode=0
Sep 11 18:13:30.659008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.658621    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"
Sep 11 18:13:30.659008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.658664    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" event={"ID":"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58","Type":"ContainerDied","Data":"4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67"}
Sep 11 18:13:30.659008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.658720    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb" event={"ID":"8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58","Type":"ContainerDied","Data":"3a33b920e878cb915602dd9c0e6603e6ef7d194c969c3a2c524fef8ef7c5ee9b"}
Sep 11 18:13:30.659008 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.658750    2582 scope.go:117] "RemoveContainer" containerID="4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67"
Sep 11 18:13:30.668493 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.668471    2582 scope.go:117] "RemoveContainer" containerID="39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"
Sep 11 18:13:30.676551 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.676531    2582 scope.go:117] "RemoveContainer" containerID="4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67"
Sep 11 18:13:30.676778 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:13:30.676761    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67\": container with ID starting with 4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67 not found: ID does not exist" containerID="4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67"
Sep 11 18:13:30.676830 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.676785    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67"} err="failed to get container status \"4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67\": rpc error: code = NotFound desc = could not find container \"4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67\": container with ID starting with 4064d1ed8f7d6fd313bd5f1c3911b2fafccaea7bbb2646f8047ac28293ad1d67 not found: ID does not exist"
Sep 11 18:13:30.676830 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.676806    2582 scope.go:117] "RemoveContainer" containerID="39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"
Sep 11 18:13:30.677071 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:13:30.677050    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d\": container with ID starting with 39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d not found: ID does not exist" containerID="39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"
Sep 11 18:13:30.677122 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.677077    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d"} err="failed to get container status \"39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d\": rpc error: code = NotFound desc = could not find container \"39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d\": container with ID starting with 39e4a38a20655f68b99d8ee7aa6b9bd5e1b67b0b8b67f050ffd5fdcbee08177d not found: ID does not exist"
Sep 11 18:13:30.680128 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.680106    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"]
Sep 11 18:13:30.685988 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:30.685967    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-1bf937-openshift-default-55cb5d6fc4-g7fxb"]
Sep 11 18:13:31.747258 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:31.747223    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" path="/var/lib/kubelet/pods/8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58/volumes"
Sep 11 18:13:37.765159 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:37.765124    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:13:37.778753 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:37.778723    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:13:38.221437 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:38.221343    2582 scope.go:117] "RemoveContainer" containerID="f629aff52fa018fa3b67c7558acc6fc7c2201716432726abf6eede24e67a65a4"
Sep 11 18:13:44.394671 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:13:44.394633    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:30.791253 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:14:30.791163    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:39.746494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:14:39.746444    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:49.754284 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:14:49.754243    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:14:50.098644 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:14:50.098560    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:14:51.443537 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:14:51.443497    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:00.132676 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.132643    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"]
Sep 11 18:15:00.133051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133020    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy"
Sep 11 18:15:00.133051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133031    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy"
Sep 11 18:15:00.133051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133046    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56c4f1fe-98e6-4d26-82ea-76ef081efbb9" containerName="revoke-keys"
Sep 11 18:15:00.133051 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133052    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="56c4f1fe-98e6-4d26-82ea-76ef081efbb9" containerName="revoke-keys"
Sep 11 18:15:00.133195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133069    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy"
Sep 11 18:15:00.133195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133074    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy"
Sep 11 18:15:00.133195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133151    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="56c4f1fe-98e6-4d26-82ea-76ef081efbb9" containerName="revoke-keys"
Sep 11 18:15:00.133195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133158    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy"
Sep 11 18:15:00.133195 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.133165    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c9dea0c-e3c3-4e35-929d-fa5ec0ccad58" containerName="istio-proxy"
Sep 11 18:15:00.136254 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.136236    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"
Sep 11 18:15:00.138851 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.138826    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zz2hp\""
Sep 11 18:15:00.146034 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.146009    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"
Sep 11 18:15:00.148460 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.148427    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"]
Sep 11 18:15:00.417532 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:00.417511    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"]
Sep 11 18:15:00.419812 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:15:00.419788    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ba83d5_f6a9_4246_8b31_0cc1e43469cd.slice/crio-991b21dd3207eec80d9a37fae2e352530629cb240d95ea44a5dc8e1606642040 WatchSource:0}: Error finding container 991b21dd3207eec80d9a37fae2e352530629cb240d95ea44a5dc8e1606642040: Status 404 returned error can't find the container with id 991b21dd3207eec80d9a37fae2e352530629cb240d95ea44a5dc8e1606642040
Sep 11 18:15:01.128046 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:01.128014    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz" event={"ID":"86ba83d5-f6a9-4246-8b31-0cc1e43469cd","Type":"ContainerStarted","Data":"9dd3bad029aba20e42c93826d2ca8e1410b78ce8125a03aebb35a8f3448e67aa"}
Sep 11 18:15:01.128225 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:01.128062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz" event={"ID":"86ba83d5-f6a9-4246-8b31-0cc1e43469cd","Type":"ContainerStarted","Data":"991b21dd3207eec80d9a37fae2e352530629cb240d95ea44a5dc8e1606642040"}
Sep 11 18:15:01.148124 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:01.148069    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz" podStartSLOduration=1.148052852 podStartE2EDuration="1.148052852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:15:01.142495709 +0000 UTC m=+2184.012538656" watchObservedRunningTime="2026-09-11 18:15:01.148052852 +0000 UTC m=+2184.018095795"
Sep 11 18:15:02.134949 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:02.134911    2582 generic.go:356] "Generic (PLEG): container finished" podID="86ba83d5-f6a9-4246-8b31-0cc1e43469cd" containerID="9dd3bad029aba20e42c93826d2ca8e1410b78ce8125a03aebb35a8f3448e67aa" exitCode=0
Sep 11 18:15:02.134949 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:02.134952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz" event={"ID":"86ba83d5-f6a9-4246-8b31-0cc1e43469cd","Type":"ContainerDied","Data":"9dd3bad029aba20e42c93826d2ca8e1410b78ce8125a03aebb35a8f3448e67aa"}
Sep 11 18:15:03.275106 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:03.275083    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"
Sep 11 18:15:04.147028 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:04.146943    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz" event={"ID":"86ba83d5-f6a9-4246-8b31-0cc1e43469cd","Type":"ContainerDied","Data":"991b21dd3207eec80d9a37fae2e352530629cb240d95ea44a5dc8e1606642040"}
Sep 11 18:15:04.147028 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:04.146995    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="991b21dd3207eec80d9a37fae2e352530629cb240d95ea44a5dc8e1606642040"
Sep 11 18:15:04.147028 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:04.146957    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29819175-r8fdz"
Sep 11 18:15:04.218970 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:04.218936    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:04.317076 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:04.317041    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:15:15.823787 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:15.823755    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:15:23.411785 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:23.411752    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:15:30.317379 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:30.317328    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:15:36.606981 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:36.606947    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:15:49.812155 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:49.812116    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:15:58.005104 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:15:58.005066    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:16:05.234399 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.234309    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"]
Sep 11 18:16:05.234803 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.234534    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x" podUID="89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" containerName="manager" containerID="cri-o://77e41d4ce950e21d8727680b69de3c4a54b149d21040efeca7cde235793bce66" gracePeriod=10
Sep 11 18:16:05.467329 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.467299    2582 generic.go:356] "Generic (PLEG): container finished" podID="89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" containerID="77e41d4ce950e21d8727680b69de3c4a54b149d21040efeca7cde235793bce66" exitCode=0
Sep 11 18:16:05.467508 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.467379    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x" event={"ID":"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1","Type":"ContainerDied","Data":"77e41d4ce950e21d8727680b69de3c4a54b149d21040efeca7cde235793bce66"}
Sep 11 18:16:05.499554 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.499498    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 18:16:05.611275 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.611245    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-extensions-socket-volume\") pod \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") "
Sep 11 18:16:05.611483 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.611375    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdssb\" (UniqueName: \"kubernetes.io/projected/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-kube-api-access-mdssb\") pod \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\" (UID: \"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1\") "
Sep 11 18:16:05.611673 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.611648    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" (UID: "89d4eb8d-7a76-445f-beac-d9cc2a92c3c1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:16:05.613411 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.613390    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-kube-api-access-mdssb" (OuterVolumeSpecName: "kube-api-access-mdssb") pod "89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" (UID: "89d4eb8d-7a76-445f-beac-d9cc2a92c3c1"). InnerVolumeSpecName "kube-api-access-mdssb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:16:05.712422 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.712391    2582 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-extensions-socket-volume\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:16:05.712422 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:05.712416    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdssb\" (UniqueName: \"kubernetes.io/projected/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1-kube-api-access-mdssb\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:16:06.473575 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:06.473543    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"
Sep 11 18:16:06.474040 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:06.473593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x" event={"ID":"89d4eb8d-7a76-445f-beac-d9cc2a92c3c1","Type":"ContainerDied","Data":"1db7800356da85c6837800f056c320e662a345e540d6eb005a9c12e3c5f27d69"}
Sep 11 18:16:06.474040 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:06.473662    2582 scope.go:117] "RemoveContainer" containerID="77e41d4ce950e21d8727680b69de3c4a54b149d21040efeca7cde235793bce66"
Sep 11 18:16:06.494430 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:06.494405    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"]
Sep 11 18:16:06.501117 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:06.501095    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-7vl8x"]
Sep 11 18:16:07.747012 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:16:07.746979    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" path="/var/lib/kubelet/pods/89d4eb8d-7a76-445f-beac-d9cc2a92c3c1/volumes"
Sep 11 18:17:06.024103 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024066    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"]
Sep 11 18:17:06.024737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024522    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86ba83d5-f6a9-4246-8b31-0cc1e43469cd" containerName="cleanup"
Sep 11 18:17:06.024737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024538    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="86ba83d5-f6a9-4246-8b31-0cc1e43469cd" containerName="cleanup"
Sep 11 18:17:06.024737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024568    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" containerName="manager"
Sep 11 18:17:06.024737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024574    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" containerName="manager"
Sep 11 18:17:06.024737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024631    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="86ba83d5-f6a9-4246-8b31-0cc1e43469cd" containerName="cleanup"
Sep 11 18:17:06.024737 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.024638    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d4eb8d-7a76-445f-beac-d9cc2a92c3c1" containerName="manager"
Sep 11 18:17:06.027758 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.027738    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.030606 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.030582    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-jcg7f\""
Sep 11 18:17:06.045635 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.045606    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"]
Sep 11 18:17:06.124812 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.124776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89xmm\" (UniqueName: \"kubernetes.io/projected/b64525b6-dc51-4983-a0e7-b6c5134e234c-kube-api-access-89xmm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4pht6\" (UID: \"b64525b6-dc51-4983-a0e7-b6c5134e234c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.124960 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.124858    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b64525b6-dc51-4983-a0e7-b6c5134e234c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4pht6\" (UID: \"b64525b6-dc51-4983-a0e7-b6c5134e234c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.225822 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.225789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89xmm\" (UniqueName: \"kubernetes.io/projected/b64525b6-dc51-4983-a0e7-b6c5134e234c-kube-api-access-89xmm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4pht6\" (UID: \"b64525b6-dc51-4983-a0e7-b6c5134e234c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.225970 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.225856    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b64525b6-dc51-4983-a0e7-b6c5134e234c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4pht6\" (UID: \"b64525b6-dc51-4983-a0e7-b6c5134e234c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.226227 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.226208    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b64525b6-dc51-4983-a0e7-b6c5134e234c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4pht6\" (UID: \"b64525b6-dc51-4983-a0e7-b6c5134e234c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.236271 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.236248    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89xmm\" (UniqueName: \"kubernetes.io/projected/b64525b6-dc51-4983-a0e7-b6c5134e234c-kube-api-access-89xmm\") pod \"kuadrant-operator-controller-manager-5769f99d9f-4pht6\" (UID: \"b64525b6-dc51-4983-a0e7-b6c5134e234c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.338378 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.338294    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.619801 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.619777    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"]
Sep 11 18:17:06.621330 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:17:06.621305    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb64525b6_dc51_4983_a0e7_b6c5134e234c.slice/crio-f92a1372716053d1e813ce278dbb67fb1a31348c6f1ee38e8524829d8201c30e WatchSource:0}: Error finding container f92a1372716053d1e813ce278dbb67fb1a31348c6f1ee38e8524829d8201c30e: Status 404 returned error can't find the container with id f92a1372716053d1e813ce278dbb67fb1a31348c6f1ee38e8524829d8201c30e
Sep 11 18:17:06.788290 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.788252    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6" event={"ID":"b64525b6-dc51-4983-a0e7-b6c5134e234c","Type":"ContainerStarted","Data":"1c29d18316fe3bd24a59478953f1a59448221048cec285bda4a05d252d27a653"}
Sep 11 18:17:06.788501 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.788312    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6" event={"ID":"b64525b6-dc51-4983-a0e7-b6c5134e234c","Type":"ContainerStarted","Data":"f92a1372716053d1e813ce278dbb67fb1a31348c6f1ee38e8524829d8201c30e"}
Sep 11 18:17:06.788501 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.788376    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:06.813187 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:06.813141    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6" podStartSLOduration=1.813126835 podStartE2EDuration="1.813126835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:17:06.812074383 +0000 UTC m=+2309.682117339" watchObservedRunningTime="2026-09-11 18:17:06.813126835 +0000 UTC m=+2309.683169779"
Sep 11 18:17:07.871609 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:07.871570    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:17.796246 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:17.796213    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-4pht6"
Sep 11 18:17:26.809142 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:26.809102    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:17:33.511412 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:33.511315    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:17:38.273980 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:38.273949    2582 scope.go:117] "RemoveContainer" containerID="aeead91dc516ddcc3cab4948f144d20743a097ded3259e20507f3124816b696a"
Sep 11 18:17:38.285200 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:38.283834    2582 scope.go:117] "RemoveContainer" containerID="1adeb14786a9b5b815d0274bfc0c60940ac0fb655c452e5df826d8717530ba8c"
Sep 11 18:17:42.503751 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:42.503712    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:17:59.378394 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.378365    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"]
Sep 11 18:17:59.381971 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.381949    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.388672 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.388646    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-c0ed78ba-kserve-self-signed-certs\""
Sep 11 18:17:59.396515 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.396491    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"]
Sep 11 18:17:59.418848 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.418820    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:17:59.482237 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.482208    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tmp-dir\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.482237 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.482238    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tls-certs\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.482461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.482257    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-model-cache\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.482461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.482322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh44x\" (UniqueName: \"kubernetes.io/projected/19a03120-2c54-4c1e-9416-56b1d4f94cbc-kube-api-access-qh44x\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.482461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.482386    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-dshm\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.482461 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.482403    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-home\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.583723 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.583692    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-dshm\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.583723 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.583728    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-home\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.583910 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.583865    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tmp-dir\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.583910 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.583897    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tls-certs\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.584014 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.583922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-model-cache\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.584014 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.583989    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh44x\" (UniqueName: \"kubernetes.io/projected/19a03120-2c54-4c1e-9416-56b1d4f94cbc-kube-api-access-qh44x\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.584125 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.584073    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-home\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.584246 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.584216    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tmp-dir\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.584417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.584275    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-model-cache\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.585973 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.585951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-dshm\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.586407 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.586387    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tls-certs\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.591749 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.591725    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh44x\" (UniqueName: \"kubernetes.io/projected/19a03120-2c54-4c1e-9416-56b1d4f94cbc-kube-api-access-qh44x\") pod \"e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") " pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.691698 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.691622    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:17:59.966473 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:17:59.966447    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"]
Sep 11 18:17:59.968447 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:17:59.968421    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19a03120_2c54_4c1e_9416_56b1d4f94cbc.slice/crio-05d9830132009fff81a7c5b98884be67a69d8ac00b7098c4931871c0e055099e WatchSource:0}: Error finding container 05d9830132009fff81a7c5b98884be67a69d8ac00b7098c4931871c0e055099e: Status 404 returned error can't find the container with id 05d9830132009fff81a7c5b98884be67a69d8ac00b7098c4931871c0e055099e
Sep 11 18:18:00.061284 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:00.061213    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn" event={"ID":"19a03120-2c54-4c1e-9416-56b1d4f94cbc","Type":"ContainerStarted","Data":"ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd"}
Sep 11 18:18:00.061483 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:00.061290    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn" event={"ID":"19a03120-2c54-4c1e-9416-56b1d4f94cbc","Type":"ContainerStarted","Data":"05d9830132009fff81a7c5b98884be67a69d8ac00b7098c4931871c0e055099e"}
Sep 11 18:18:00.081851 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:00.081805    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn" podStartSLOduration=1.081789222 podStartE2EDuration="1.081789222s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:17:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:18:00.080990884 +0000 UTC m=+2362.951033830" watchObservedRunningTime="2026-09-11 18:18:00.081789222 +0000 UTC m=+2362.951832166"
Sep 11 18:18:01.843981 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:01.843942    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:18:09.692061 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:09.692027    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:18:09.692662 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:09.692091    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:18:09.704549 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:09.704509    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:18:09.729430 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:09.729401    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:10.122335 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:10.122303    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:18:12.605247 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:12.605211    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:18:24.823557 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:24.823517    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"]
Sep 11 18:18:24.870983 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:24.870942    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:25.928027 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:25.927993    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:27.906773 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:27.906735    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:18:36.509966 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:36.509926    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:18:37.802342 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:37.802313    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:18:37.811525 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:37.811503    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:18:38.305994 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:38.305957    2582 scope.go:117] "RemoveContainer" containerID="ba646a895afe72f266e1491ab878bff7c0d0f719c6265a29b3d06658c98fe4c8"
Sep 11 18:18:39.340842 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:39.340812    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:18:39.842823 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:39.842787    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn" podUID="19a03120-2c54-4c1e-9416-56b1d4f94cbc" containerName="main" containerID="cri-o://ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd" gracePeriod=45
Sep 11 18:18:40.097948 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.097883    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:18:40.132761 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.132728    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tls-certs\") pod \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") "
Sep 11 18:18:40.132923 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.132801    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-home\") pod \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") "
Sep 11 18:18:40.132923 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.132867    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh44x\" (UniqueName: \"kubernetes.io/projected/19a03120-2c54-4c1e-9416-56b1d4f94cbc-kube-api-access-qh44x\") pod \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") "
Sep 11 18:18:40.132923 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.132905    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-dshm\") pod \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") "
Sep 11 18:18:40.133103 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.132929    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tmp-dir\") pod \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") "
Sep 11 18:18:40.133103 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.132951    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-model-cache\") pod \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\" (UID: \"19a03120-2c54-4c1e-9416-56b1d4f94cbc\") "
Sep 11 18:18:40.133103 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.133035    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-home" (OuterVolumeSpecName: "home") pod "19a03120-2c54-4c1e-9416-56b1d4f94cbc" (UID: "19a03120-2c54-4c1e-9416-56b1d4f94cbc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:40.133273 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.133180    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "19a03120-2c54-4c1e-9416-56b1d4f94cbc" (UID: "19a03120-2c54-4c1e-9416-56b1d4f94cbc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:40.133273 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.133218    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-model-cache" (OuterVolumeSpecName: "model-cache") pod "19a03120-2c54-4c1e-9416-56b1d4f94cbc" (UID: "19a03120-2c54-4c1e-9416-56b1d4f94cbc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:40.133417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.133336    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-home\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:18:40.133417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.133371    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tmp-dir\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:18:40.133417 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.133385    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-model-cache\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:18:40.135084 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.135053    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-dshm" (OuterVolumeSpecName: "dshm") pod "19a03120-2c54-4c1e-9416-56b1d4f94cbc" (UID: "19a03120-2c54-4c1e-9416-56b1d4f94cbc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 18:18:40.135084 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.135071    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "19a03120-2c54-4c1e-9416-56b1d4f94cbc" (UID: "19a03120-2c54-4c1e-9416-56b1d4f94cbc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 18:18:40.135230 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.135142    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19a03120-2c54-4c1e-9416-56b1d4f94cbc-kube-api-access-qh44x" (OuterVolumeSpecName: "kube-api-access-qh44x") pod "19a03120-2c54-4c1e-9416-56b1d4f94cbc" (UID: "19a03120-2c54-4c1e-9416-56b1d4f94cbc"). InnerVolumeSpecName "kube-api-access-qh44x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 18:18:40.234678 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.234646    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh44x\" (UniqueName: \"kubernetes.io/projected/19a03120-2c54-4c1e-9416-56b1d4f94cbc-kube-api-access-qh44x\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:18:40.234678 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.234673    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/19a03120-2c54-4c1e-9416-56b1d4f94cbc-dshm\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:18:40.234678 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.234683    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/19a03120-2c54-4c1e-9416-56b1d4f94cbc-tls-certs\") on node \"ip-10-0-128-237.ec2.internal\" DevicePath \"\""
Sep 11 18:18:40.273071 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.273038    2582 generic.go:356] "Generic (PLEG): container finished" podID="19a03120-2c54-4c1e-9416-56b1d4f94cbc" containerID="ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd" exitCode=0
Sep 11 18:18:40.273201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.273080    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"
Sep 11 18:18:40.273201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.273114    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn" event={"ID":"19a03120-2c54-4c1e-9416-56b1d4f94cbc","Type":"ContainerDied","Data":"ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd"}
Sep 11 18:18:40.273201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.273154    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn" event={"ID":"19a03120-2c54-4c1e-9416-56b1d4f94cbc","Type":"ContainerDied","Data":"05d9830132009fff81a7c5b98884be67a69d8ac00b7098c4931871c0e055099e"}
Sep 11 18:18:40.273201 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.273171    2582 scope.go:117] "RemoveContainer" containerID="ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd"
Sep 11 18:18:40.282827 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.282810    2582 scope.go:117] "RemoveContainer" containerID="ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd"
Sep 11 18:18:40.283078 ip-10-0-128-237 kubenswrapper[2582]: E0911 18:18:40.283060    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd\": container with ID starting with ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd not found: ID does not exist" containerID="ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd"
Sep 11 18:18:40.283150 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.283084    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd"} err="failed to get container status \"ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd\": rpc error: code = NotFound desc = could not find container \"ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd\": container with ID starting with ffbf93e473b22919c4c2ee0630ee3afc675bbf2e58281ff5a172ccad8ffa2bcd not found: ID does not exist"
Sep 11 18:18:40.298917 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.298894    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"]
Sep 11 18:18:40.303259 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:40.303239    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-c0ed78ba-kserve-5d6fc4f6c-h9csn"]
Sep 11 18:18:41.748148 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:41.748113    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19a03120-2c54-4c1e-9416-56b1d4f94cbc" path="/var/lib/kubelet/pods/19a03120-2c54-4c1e-9416-56b1d4f94cbc/volumes"
Sep 11 18:18:44.307464 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:44.307432    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:18:52.132679 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:52.132642    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:18:59.113404 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:18:59.113364    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:19:04.809331 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:04.809247    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:19:11.704935 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:11.704899    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-bg4v4"]
Sep 11 18:19:20.631468 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:20.631429    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pw7kj_f483e181-6ffb-4ba2-b50a-c622583c846b/manager/0.log"
Sep 11 18:19:21.581149 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:21.581122    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-m5wb2_75971c72-da6f-40e6-ac3b-ea87b6151779/copy-manifests/0.log"
Sep 11 18:19:21.589392 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:21.589361    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-m5wb2_75971c72-da6f-40e6-ac3b-ea87b6151779/manager/0.log"
Sep 11 18:19:22.198889 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:22.198862    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819160-6gstt_44da4928-5a19-40bf-874d-df9d180b51de/cleanup/0.log"
Sep 11 18:19:22.319494 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:22.319466    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29819175-r8fdz_86ba83d5-f6a9-4246-8b31-0cc1e43469cd/cleanup/0.log"
Sep 11 18:19:22.438267 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:22.438239    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-cv6vq_98bbe7f0-81b9-4934-ad6f-7b09595fc1b7/postgres/0.log"
Sep 11 18:19:23.439650 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:23.439617    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-gbxbq_0ce262a3-df86-49d9-ab72-6eab8d280c04/manager/0.log"
Sep 11 18:19:23.571832 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:23.571806    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-wzvc8_26d8b7e2-8501-47f8-8e5c-9056b155a849/manager/0.log"
Sep 11 18:19:23.697101 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:23.697028    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-kn68h_32a132e8-ee3c-42f4-bd77-fd2889af3040/kuadrant-console-plugin/0.log"
Sep 11 18:19:23.840824 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:23.840797    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-4pht6_b64525b6-dc51-4983-a0e7-b6c5134e234c/manager/0.log"
Sep 11 18:19:23.987168 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:23.987138    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-bg4v4_2ada2023-141e-4e73-abfd-8abc6c4b9169/limitador/0.log"
Sep 11 18:19:24.886339 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:24.886307    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-fwh9z_a9af83e1-74b6-4d75-a854-1d49d829d2da/kube-auth-proxy/0.log"
Sep 11 18:19:25.153159 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:25.153077    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-76smh_ecd77024-ebaf-4b62-8ff2-ace294c3147e/payload-pre-processing/0.log"
Sep 11 18:19:25.287053 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:25.287020    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v8vwn_ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844/payload-processing/0.log"
Sep 11 18:19:25.413478 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:25.413407    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89b8b6fbd-68pzd_69bfb1df-6b20-422a-9264-bd89506ab8d4/router/0.log"
Sep 11 18:19:26.028565 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:26.028532    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-gv7zk_68dff64a-cf98-47ea-a3da-d71725078cf3/main/0.log"
Sep 11 18:19:26.277552 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:26.277521    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fcwpd7_31478caa-4a36-4fad-9dd7-93c88b2964d5/main/0.log"
Sep 11 18:19:32.917812 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.917779    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9jhl8/must-gather-p7hbd"]
Sep 11 18:19:32.918182 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.918169    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19a03120-2c54-4c1e-9416-56b1d4f94cbc" containerName="main"
Sep 11 18:19:32.918182 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.918183    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="19a03120-2c54-4c1e-9416-56b1d4f94cbc" containerName="main"
Sep 11 18:19:32.918270 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.918261    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="19a03120-2c54-4c1e-9416-56b1d4f94cbc" containerName="main"
Sep 11 18:19:32.921533 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.921518    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:32.925342 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.925319    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jhl8\"/\"kube-root-ca.crt\""
Sep 11 18:19:32.925677 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.925655    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9jhl8\"/\"default-dockercfg-d7m62\""
Sep 11 18:19:32.925769 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.925661    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9jhl8\"/\"openshift-service-ca.crt\""
Sep 11 18:19:32.939284 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.939260    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jhl8/must-gather-p7hbd"]
Sep 11 18:19:32.970622 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.970594    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:32.994517 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.994486    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nr882\" (UniqueName: \"kubernetes.io/projected/32675953-e8c7-4216-a501-21512bfc96f5-kube-api-access-nr882\") pod \"must-gather-p7hbd\" (UID: \"32675953-e8c7-4216-a501-21512bfc96f5\") " pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:32.994639 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:32.994532    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/32675953-e8c7-4216-a501-21512bfc96f5-must-gather-output\") pod \"must-gather-p7hbd\" (UID: \"32675953-e8c7-4216-a501-21512bfc96f5\") " pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:33.095448 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.095412    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/32675953-e8c7-4216-a501-21512bfc96f5-must-gather-output\") pod \"must-gather-p7hbd\" (UID: \"32675953-e8c7-4216-a501-21512bfc96f5\") " pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:33.095617 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.095524    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nr882\" (UniqueName: \"kubernetes.io/projected/32675953-e8c7-4216-a501-21512bfc96f5-kube-api-access-nr882\") pod \"must-gather-p7hbd\" (UID: \"32675953-e8c7-4216-a501-21512bfc96f5\") " pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:33.095769 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.095747    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/32675953-e8c7-4216-a501-21512bfc96f5-must-gather-output\") pod \"must-gather-p7hbd\" (UID: \"32675953-e8c7-4216-a501-21512bfc96f5\") " pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:33.106254 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.106228    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nr882\" (UniqueName: \"kubernetes.io/projected/32675953-e8c7-4216-a501-21512bfc96f5-kube-api-access-nr882\") pod \"must-gather-p7hbd\" (UID: \"32675953-e8c7-4216-a501-21512bfc96f5\") " pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:33.230597 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.230576    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jhl8/must-gather-p7hbd"
Sep 11 18:19:33.505164 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.505140    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jhl8/must-gather-p7hbd"]
Sep 11 18:19:33.506973 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:19:33.506945    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32675953_e8c7_4216_a501_21512bfc96f5.slice/crio-72bae0577a5c95c7ccc88e0dedd913ef3396c7d975a0d4c5fac545db22fff70d WatchSource:0}: Error finding container 72bae0577a5c95c7ccc88e0dedd913ef3396c7d975a0d4c5fac545db22fff70d: Status 404 returned error can't find the container with id 72bae0577a5c95c7ccc88e0dedd913ef3396c7d975a0d4c5fac545db22fff70d
Sep 11 18:19:33.508907 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.508887    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 18:19:33.544416 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:33.544378    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/must-gather-p7hbd" event={"ID":"32675953-e8c7-4216-a501-21512bfc96f5","Type":"ContainerStarted","Data":"72bae0577a5c95c7ccc88e0dedd913ef3396c7d975a0d4c5fac545db22fff70d"}
Sep 11 18:19:34.306741 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:34.306714    2582 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log" limit=1024
Sep 11 18:19:34.555063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:34.554973    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/must-gather-p7hbd" event={"ID":"32675953-e8c7-4216-a501-21512bfc96f5","Type":"ContainerStarted","Data":"3a2694e95c8bf113abdad345708934e8a836d36dad8d08355abfb3d85a1dd0a5"}
Sep 11 18:19:34.555063 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:34.555039    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/must-gather-p7hbd" event={"ID":"32675953-e8c7-4216-a501-21512bfc96f5","Type":"ContainerStarted","Data":"2de86cd38962c5f87c4ad556d48c670b4dac782fe3125524f9162f7879b416f3"}
Sep 11 18:19:34.572949 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:34.572899    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9jhl8/must-gather-p7hbd" podStartSLOduration=1.777205365 podStartE2EDuration="2.572881676s" totalImagesPullingTime="795.676311ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 18:19:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:34.571403082 +0000 UTC m=+2457.441446027" watchObservedRunningTime="2026-09-11 18:19:34.572881676 +0000 UTC m=+2457.442924686"
Sep 11 18:19:36.000819 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:36.000778    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-cgzbn_59f9e5b9-528f-4400-a787-3c339153c3b4/global-pull-secret-syncer/0.log"
Sep 11 18:19:36.112836 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:36.112804    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7c6kr_9cc0998d-3386-4712-a688-acd2e7b04366/konnectivity-agent/0.log"
Sep 11 18:19:36.237608 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:36.237577    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-237.ec2.internal_8c6603c74816e2e6b08fd8fe02fdf018/haproxy/0.log"
Sep 11 18:19:38.326157 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:38.326117    2582 scope.go:117] "RemoveContainer" containerID="a7e69948c427be99c209b9a3686ab64758d5c11e30885b529e6e1884beec118e"
Sep 11 18:19:38.342655 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:38.342321    2582 scope.go:117] "RemoveContainer" containerID="977f7e1b5868d7b3a2a7f3092c5bca9b5c4022d376df568189feca92f44b03e0"
Sep 11 18:19:40.035823 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:40.035784    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-gbxbq_0ce262a3-df86-49d9-ab72-6eab8d280c04/manager/0.log"
Sep 11 18:19:40.068368 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:40.068324    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-wzvc8_26d8b7e2-8501-47f8-8e5c-9056b155a849/manager/0.log"
Sep 11 18:19:40.119317 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:40.119271    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-kn68h_32a132e8-ee3c-42f4-bd77-fd2889af3040/kuadrant-console-plugin/0.log"
Sep 11 18:19:40.226040 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:40.225999    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-4pht6_b64525b6-dc51-4983-a0e7-b6c5134e234c/manager/0.log"
Sep 11 18:19:40.276058 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:40.275993    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-bg4v4_2ada2023-141e-4e73-abfd-8abc6c4b9169/limitador/0.log"
Sep 11 18:19:42.323669 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:42.323629    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cd954b897-p6tmp_921d4454-3d1d-489b-a9d6-31f5e8af7dc5/cluster-monitoring-operator/0.log"
Sep 11 18:19:42.519612 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:42.519582    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9vxts_bab029ac-f4d5-400f-9b28-bbbf0b17a5f4/node-exporter/0.log"
Sep 11 18:19:42.558438 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:42.558405    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9vxts_bab029ac-f4d5-400f-9b28-bbbf0b17a5f4/kube-rbac-proxy/0.log"
Sep 11 18:19:42.587067 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:42.586989    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9vxts_bab029ac-f4d5-400f-9b28-bbbf0b17a5f4/init-textfile/0.log"
Sep 11 18:19:45.117556 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:45.117514    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-qwtvf_041f6f00-15f9-48cf-82a2-fe253ff1f70a/networking-console-plugin/0.log"
Sep 11 18:19:45.813659 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:45.813592    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/1.log"
Sep 11 18:19:45.822158 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:45.822120    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-696884fb9c-bnb6l_cd176d7f-b655-4a6a-90d3-7df96b78de54/console-operator/2.log"
Sep 11 18:19:46.220073 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.219973    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"]
Sep 11 18:19:46.227113 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.227083    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.239916 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.239885    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"]
Sep 11 18:19:46.343540 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.343496    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-sys\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.343734 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.343567    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-podres\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.343734 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.343637    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6gtk\" (UniqueName: \"kubernetes.io/projected/15915ec0-d5f4-452c-afbc-28eb50691412-kube-api-access-w6gtk\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.343734 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.343683    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-lib-modules\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.343734 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.343731    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-proc\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445243 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445198    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-sys\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445447 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-podres\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445447 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6gtk\" (UniqueName: \"kubernetes.io/projected/15915ec0-d5f4-452c-afbc-28eb50691412-kube-api-access-w6gtk\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445447 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445399    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-lib-modules\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445646 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445450    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-proc\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445646 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445571    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-proc\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445646 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445635    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-sys\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.445820 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.445749    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-podres\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.446209 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.446179    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/15915ec0-d5f4-452c-afbc-28eb50691412-lib-modules\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.458104 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.458076    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6gtk\" (UniqueName: \"kubernetes.io/projected/15915ec0-d5f4-452c-afbc-28eb50691412-kube-api-access-w6gtk\") pod \"perf-node-gather-daemonset-w7d86\" (UID: \"15915ec0-d5f4-452c-afbc-28eb50691412\") " pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.546246 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.546209    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:46.552830 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.552805    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-qmqz9_0579203d-ba0e-40e8-9756-6942bed7ec3a/download-server/0.log"
Sep 11 18:19:46.843893 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:46.843863    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"]
Sep 11 18:19:46.845633 ip-10-0-128-237 kubenswrapper[2582]: W0911 18:19:46.845604    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod15915ec0_d5f4_452c_afbc_28eb50691412.slice/crio-c310d35d9e0ede1d913781e9854f9d1ac4c8c6cf860c5a2be2e8378e2b2af16b WatchSource:0}: Error finding container c310d35d9e0ede1d913781e9854f9d1ac4c8c6cf860c5a2be2e8378e2b2af16b: Status 404 returned error can't find the container with id c310d35d9e0ede1d913781e9854f9d1ac4c8c6cf860c5a2be2e8378e2b2af16b
Sep 11 18:19:47.416831 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:47.416785    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-86695b7cc7-nvxnn_a8cb7ac8-7ea0-48db-94bc-49f00c7f1eeb/volume-data-source-validator/0.log"
Sep 11 18:19:47.673235 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:47.673143    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86" event={"ID":"15915ec0-d5f4-452c-afbc-28eb50691412","Type":"ContainerStarted","Data":"5e514bba6557b03a671c14c9a0fc0d65aec3d4a6ea91985b93ab7791c828d143"}
Sep 11 18:19:47.673421 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:47.673249    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86" event={"ID":"15915ec0-d5f4-452c-afbc-28eb50691412","Type":"ContainerStarted","Data":"c310d35d9e0ede1d913781e9854f9d1ac4c8c6cf860c5a2be2e8378e2b2af16b"}
Sep 11 18:19:47.673491 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:47.673412    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:47.697897 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:47.697836    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86" podStartSLOduration=1.697814975 podStartE2EDuration="1.697814975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 18:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 18:19:47.694508928 +0000 UTC m=+2470.564551885" watchObservedRunningTime="2026-09-11 18:19:47.697814975 +0000 UTC m=+2470.567857923"
Sep 11 18:19:48.793199 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:48.793170    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-545rn_fc070bb0-e27e-45c8-b7e5-987dabdbef97/dns/0.log"
Sep 11 18:19:48.822767 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:48.822741    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-545rn_fc070bb0-e27e-45c8-b7e5-987dabdbef97/kube-rbac-proxy/0.log"
Sep 11 18:19:48.959810 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:48.959782    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vv7sb_783d55b1-3f78-4435-9d56-cb60f6f7c2f3/dns-node-resolver/0.log"
Sep 11 18:19:49.641799 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:49.641772    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xx9jz_f6fe38a1-c4b8-4cdd-8156-27a378e522f7/node-ca/0.log"
Sep 11 18:19:51.567548 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:51.567517    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-fwh9z_a9af83e1-74b6-4d75-a854-1d49d829d2da/kube-auth-proxy/0.log"
Sep 11 18:19:51.648132 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:51.648098    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-76smh_ecd77024-ebaf-4b62-8ff2-ace294c3147e/payload-pre-processing/0.log"
Sep 11 18:19:51.690441 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:51.690410    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v8vwn_ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844/payload-processing/0.log"
Sep 11 18:19:51.721235 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:51.721193    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89b8b6fbd-68pzd_69bfb1df-6b20-422a-9264-bd89506ab8d4/router/0.log"
Sep 11 18:19:52.683618 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:52.683581    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zzp4k_26fa16d6-41b9-4a5f-b1f9-b7b0c65c63b1/serve-healthcheck-canary/0.log"
Sep 11 18:19:53.161514 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:53.161479    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-ht8lt_7a040f9a-b722-4aa6-9cf3-371f02ee172a/insights-operator/0.log"
Sep 11 18:19:53.164802 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:53.164770    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-ht8lt_7a040f9a-b722-4aa6-9cf3-371f02ee172a/insights-operator/1.log"
Sep 11 18:19:53.338540 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:53.338510    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zw2xl_d44b26f1-116a-4b7c-8a5c-15a113efe9fd/kube-rbac-proxy/0.log"
Sep 11 18:19:53.362835 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:53.362804    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zw2xl_d44b26f1-116a-4b7c-8a5c-15a113efe9fd/exporter/0.log"
Sep 11 18:19:53.389426 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:53.389395    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zw2xl_d44b26f1-116a-4b7c-8a5c-15a113efe9fd/extractor/0.log"
Sep 11 18:19:53.692072 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:53.692039    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9jhl8/perf-node-gather-daemonset-w7d86"
Sep 11 18:19:55.715088 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:55.715052    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-pw7kj_f483e181-6ffb-4ba2-b50a-c622583c846b/manager/0.log"
Sep 11 18:19:56.723992 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:56.723957    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-m5wb2_75971c72-da6f-40e6-ac3b-ea87b6151779/manager/0.log"
Sep 11 18:19:56.753149 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:56.753123    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86997d8dbf-m5wb2_75971c72-da6f-40e6-ac3b-ea87b6151779/copy-manifests/0.log"
Sep 11 18:19:57.414564 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:57.414538    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-fwh9z_a9af83e1-74b6-4d75-a854-1d49d829d2da/kube-auth-proxy/0.log"
Sep 11 18:19:57.471646 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:57.471609    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-76smh_ecd77024-ebaf-4b62-8ff2-ace294c3147e/payload-pre-processing/0.log"
Sep 11 18:19:57.507177 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:57.507143    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v8vwn_ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844/payload-processing/0.log"
Sep 11 18:19:57.527465 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:57.527438    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89b8b6fbd-68pzd_69bfb1df-6b20-422a-9264-bd89506ab8d4/router/0.log"
Sep 11 18:19:59.856369 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:59.856322    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-64d8557cb6-fwh9z_a9af83e1-74b6-4d75-a854-1d49d829d2da/kube-auth-proxy/0.log"
Sep 11 18:19:59.946305 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:59.946271    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-76smh_ecd77024-ebaf-4b62-8ff2-ace294c3147e/payload-pre-processing/0.log"
Sep 11 18:19:59.986541 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:19:59.986512    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-v8vwn_ba4e5b3c-c70f-41fa-bc34-bb5ba7d74844/payload-processing/0.log"
Sep 11 18:20:00.013980 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:20:00.013947    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-89b8b6fbd-68pzd_69bfb1df-6b20-422a-9264-bd89506ab8d4/router/0.log"
Sep 11 18:20:03.102885 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:20:03.102845    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-5mkfm_cc581405-de86-4e56-bb95-d8f7edbf1e81/migrator/0.log"
Sep 11 18:20:03.128379 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:20:03.128336    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-8b4d697c5-5mkfm_cc581405-de86-4e56-bb95-d8f7edbf1e81/graceful-termination/0.log"
Sep 11 18:20:03.498622 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:20:03.498591    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-xls2m_e471abfe-aaf5-45bb-8f10-9ec40578bf95/kube-storage-version-migrator-operator/1.log"
Sep 11 18:20:03.500432 ip-10-0-128-237 kubenswrapper[2582]: I0911 18:20:03.500395    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6c8f84f897-xls2m_e471abfe-aaf5-45bb-8f10-9ec40578bf95/kube-storage-version-migrator-operator/0.log"