Sep 10 14:11:56.440015 ip-10-0-128-41 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 14:11:56.829697 ip-10-0-128-41 kubenswrapper[2576]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:11:56.829697 ip-10-0-128-41 kubenswrapper[2576]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 14:11:56.829697 ip-10-0-128-41 kubenswrapper[2576]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:11:56.829697 ip-10-0-128-41 kubenswrapper[2576]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 14:11:56.829697 ip-10-0-128-41 kubenswrapper[2576]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:11:56.831337 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.831244    2576 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 14:11:56.834044 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834030    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:56.834044 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834044    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834048    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834052    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834055    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834058    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834061    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834064    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834067    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834070    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834073    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834076    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834079    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834082    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834084    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834087    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834094    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834097    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834099    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834101    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834104    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:56.834110 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834106    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834109    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834111    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834114    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834118    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834120    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834123    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834126    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834128    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834131    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834134    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834136    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834139    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834141    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834143    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834146    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834148    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834151    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834153    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834156    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:56.834583 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834158    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834161    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834163    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834166    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834170    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834173    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834177    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834180    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834183    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834185    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834188    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834190    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834193    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834196    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834202    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834206    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834209    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834212    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834214    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:56.835119 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834217    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834219    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834222    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834224    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834227    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834229    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834233    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834236    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834239    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834242    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834244    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834247    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834250    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834253    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834255    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834257    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834260    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834262    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834265    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:56.835574 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834267    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834269    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834272    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834274    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834277    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834279    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834282    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834284    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834288    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834291    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834293    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834296    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834298    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834300    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834303    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834305    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834307    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834310    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834312    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:56.836047 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.834315    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835001    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835007    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835010    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835013    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835016    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835020    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835022    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835025    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835028    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835030    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835033    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835035    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835038    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835040    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835043    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835046    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835048    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835051    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:56.836501 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835053    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835056    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835059    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835062    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835064    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835067    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835069    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835072    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835074    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835077    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835079    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835082    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835084    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835087    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835089    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835092    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835094    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835096    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835099    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835102    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:56.836982 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835105    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835108    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835112    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835115    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835118    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835120    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835123    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835126    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835129    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835131    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835134    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835136    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835139    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835142    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835145    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835147    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835150    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835152    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835154    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835157    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:56.837468 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835159    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835162    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835165    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835167    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835170    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835173    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835175    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835177    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835180    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835182    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835192    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835195    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835198    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835203    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835206    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835209    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835212    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835214    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835217    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:56.838013 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835220    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835223    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835226    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835228    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835232    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835234    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835238    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835243    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835245    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835248    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835251    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835253    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835256    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835258    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835261    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835263    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835266    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835268    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835271    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835273    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:56.838484 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835276    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835278    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835647    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835651    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835654    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835657    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835660    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835663    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835666    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835668    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835672    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835674    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835695    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835698    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835700    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835703    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835705    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835708    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835711    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835715    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:56.838989 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835718    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835720    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835723    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835725    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835728    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835730    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835733    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835735    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835738    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835741    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835744    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835746    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835748    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835751    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835753    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835756    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835758    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835761    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835763    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835766    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:56.839473 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835769    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835771    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835774    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835777    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835780    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835784    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835788    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835792    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835794    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835797    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835799    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835804    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835806    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835809    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835811    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835813    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835816    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835819    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835821    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:56.839971 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835824    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835826    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835828    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835831    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835834    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835836    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835839    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835842    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835845    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835847    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835850    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835852    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835854    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835860    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835863    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835866    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835868    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835870    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835873    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835875    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:56.840428 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835878    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835880    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835883    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835885    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835889    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835891    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835894    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835896    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835899    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835901    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835904    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835906    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835909    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835913    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835915    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835918    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835921    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835923    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835926    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:56.840925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835929    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835932    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.835934    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836015    2576 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836061    2576 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836069    2576 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836074    2576 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836079    2576 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836082    2576 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836086    2576 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836090    2576 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836093    2576 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836096    2576 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836100    2576 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836104    2576 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836106    2576 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836109    2576 flags.go:64] FLAG: --cgroup-root=""
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836115    2576 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836118    2576 flags.go:64] FLAG: --client-ca-file=""
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836121    2576 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836124    2576 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836128    2576 flags.go:64] FLAG: --cluster-domain=""
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836130    2576 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 14:11:56.841378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836133    2576 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836136    2576 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836140    2576 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836143    2576 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836146    2576 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836149    2576 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836152    2576 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836155    2576 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836158    2576 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836162    2576 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836165    2576 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836168    2576 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836172    2576 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836175    2576 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836178    2576 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836181    2576 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836184    2576 flags.go:64] FLAG: --enable-server="true"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836187    2576 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836191    2576 flags.go:64] FLAG: --event-burst="100"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836194    2576 flags.go:64] FLAG: --event-qps="50"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836197    2576 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836200    2576 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836203    2576 flags.go:64] FLAG: --eviction-hard=""
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836206    2576 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 14:11:56.841957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836209    2576 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836212    2576 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836216    2576 flags.go:64] FLAG: --eviction-soft=""
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836219    2576 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836221    2576 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836224    2576 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836227    2576 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836230    2576 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836233    2576 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836236    2576 flags.go:64] FLAG: --feature-gates=""
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836239    2576 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836242    2576 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836245    2576 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836248    2576 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836251    2576 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836254    2576 flags.go:64] FLAG: --help="false"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836257    2576 flags.go:64] FLAG: --hostname-override="ip-10-0-128-41.ec2.internal"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836261    2576 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836264    2576 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836267    2576 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836270    2576 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836274    2576 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836277    2576 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 14:11:56.842522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836280    2576 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836283    2576 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836286    2576 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836289    2576 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836292    2576 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836294    2576 flags.go:64] FLAG: --kube-reserved=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836297    2576 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836300    2576 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836303    2576 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836305    2576 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836308    2576 flags.go:64] FLAG: --lock-file=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836312    2576 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836315    2576 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836318    2576 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836323    2576 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836326    2576 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836328    2576 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836331    2576 flags.go:64] FLAG: --logging-format="text"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836333    2576 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836336    2576 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836339    2576 flags.go:64] FLAG: --manifest-url=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836342    2576 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836346    2576 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836349    2576 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836353    2576 flags.go:64] FLAG: --max-pods="110"
Sep 10 14:11:56.843155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836356    2576 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836360    2576 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836363    2576 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836366    2576 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836368    2576 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836372    2576 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836374    2576 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836381    2576 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836384    2576 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836387    2576 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836390    2576 flags.go:64] FLAG: --pod-cidr=""
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836393    2576 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836398    2576 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836401    2576 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836404    2576 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836407    2576 flags.go:64] FLAG: --port="10250"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836409    2576 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836412    2576 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f4f17b38877c258a"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836415    2576 flags.go:64] FLAG: --qos-reserved=""
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836419    2576 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836422    2576 flags.go:64] FLAG: --register-node="true"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836425    2576 flags.go:64] FLAG: --register-with-taints=""
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836428    2576 flags.go:64] FLAG: --registry-burst="10"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836431    2576 flags.go:64] FLAG: --registry-qps="5"
Sep 10 14:11:56.843780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836433    2576 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836436    2576 flags.go:64] FLAG: --reserved-memory=""
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836439    2576 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836443    2576 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836445    2576 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836448    2576 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836451    2576 flags.go:64] FLAG: --runonce="false"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836454    2576 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836457    2576 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836461    2576 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836464    2576 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836466    2576 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836469    2576 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836472    2576 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836475    2576 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836478    2576 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836480    2576 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836483    2576 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836485    2576 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836488    2576 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836492    2576 flags.go:64] FLAG: --system-cgroups=""
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836494    2576 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836500    2576 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836503    2576 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836506    2576 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 14:11:56.844367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836510    2576 flags.go:64] FLAG: --tls-min-version=""
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836513    2576 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836518    2576 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836521    2576 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836524    2576 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836527    2576 flags.go:64] FLAG: --v="2"
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836530    2576 flags.go:64] FLAG: --version="false"
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836534    2576 flags.go:64] FLAG: --vmodule=""
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836537    2576 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.836540    2576 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836653    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836657    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836660    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836663    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836665    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836668    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836671    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836674    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836692    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836694    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836697    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836700    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:56.845040 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836702    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836705    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836708    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836710    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836713    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836715    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836718    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836721    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836724    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836726    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836729    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836731    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836736    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836739    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836741    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836744    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836746    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836749    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836752    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836754    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:56.845705 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836757    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836759    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836762    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836765    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836767    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836771    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836775    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836778    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836781    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836783    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836786    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836788    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836791    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836794    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836796    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836798    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836801    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836803    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836806    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:56.846202 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836810    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836812    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836815    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836817    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836820    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836824    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836826    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836829    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836831    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836834    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836836    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836839    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836841    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836844    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836847    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836849    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836852    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836854    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836857    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836859    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:56.846691 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836862    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836865    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836867    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836870    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836872    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836875    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836877    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836880    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836882    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836885    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836887    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836890    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836892    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836895    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836897    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836900    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836902    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836907    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836910    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836913    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:56.847210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836916    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836918    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836921    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836923    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836926    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836930    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836933    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.836937    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.837535    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.844463    2576 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.844479    2576 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.845138    2576 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.845146    2576 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845207    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:56.847753 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845212    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845216    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845219    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845222    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845224    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845227    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845229    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845232    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845236    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845238    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845241    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845243    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845246    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845248    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845251    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845253    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845256    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845258    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845260    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845263    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:56.848126 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845265    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845268    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845271    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845273    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845276    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845278    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845281    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845284    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845287    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845290    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845293    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845295    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845298    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845300    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845303    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845305    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845308    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845310    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845313    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:56.848615 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845317    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845320    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845323    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845326    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845329    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845331    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845334    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845336    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845339    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845341    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845344    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845347    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845349    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845352    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845354    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845356    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845359    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845361    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845365    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845369    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:56.849120 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845372    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845376    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845380    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845383    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845386    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845389    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845392    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845395    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845398    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845400    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845403    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845405    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845408    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845410    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845413    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845415    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845417    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845420    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845422    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:56.849626 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845425    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845428    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845431    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845435    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845438    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845440    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845443    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845445    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845448    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845451    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845454    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845456    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845459    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845462    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845464    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845467    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845470    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845472    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845474    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:56.850115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845477    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.845482    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845608    2576 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845614    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845618    2576 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845621    2576 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845624    2576 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845627    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845630    2576 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845632    2576 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845635    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845638    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845641    2576 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845643    2576 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:11:56.850620 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845646    2576 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845648    2576 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845651    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845653    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845656    2576 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845658    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845660    2576 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845663    2576 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845665    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845668    2576 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845671    2576 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845673    2576 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845676    2576 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845698    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845703    2576 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845707    2576 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845710    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845713    2576 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845716    2576 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845718    2576 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:11:56.850997 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845721    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845724    2576 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845726    2576 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845729    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845731    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845733    2576 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845736    2576 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845738    2576 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845741    2576 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845743    2576 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845746    2576 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845748    2576 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845750    2576 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845753    2576 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845755    2576 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845758    2576 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845760    2576 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845763    2576 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845765    2576 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845768    2576 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:11:56.851500 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845771    2576 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845773    2576 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845776    2576 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845778    2576 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845781    2576 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845783    2576 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845786    2576 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845788    2576 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845792    2576 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845795    2576 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845798    2576 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845800    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845802    2576 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845805    2576 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845807    2576 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845809    2576 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845812    2576 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845814    2576 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845816    2576 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845819    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:11:56.852137 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845821    2576 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845823    2576 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845826    2576 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845830    2576 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845833    2576 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845836    2576 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845839    2576 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845843    2576 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845846    2576 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845848    2576 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845850    2576 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845853    2576 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845855    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845858    2576 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845861    2576 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845863    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845866    2576 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845868    2576 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845871    2576 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:11:56.852729 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845873    2576 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845876    2576 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845878    2576 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845881    2576 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845883    2576 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845886    2576 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845889    2576 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:11:56.845891    2576 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.845896    2576 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.846007    2576 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.847982    2576 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.848921    2576 server.go:1019] "Starting client certificate rotation"
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.849015    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:11:56.853199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.849047    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:11:56.853645 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.851291    2576 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:11:56.857488 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.857467    2576 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:11:56.867825 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.867804    2576 log.go:25] "Validated CRI v1 runtime API"
Sep 10 14:11:56.870823 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.870806    2576 log.go:25] "Validated CRI v1 image API"
Sep 10 14:11:56.871790 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.871773    2576 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 14:11:56.874081 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.874059    2576 fs.go:135] Filesystem UUIDs: map[078dddbc-5f25-432e-931d-37a497dec8cf:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8d6d61a6-04a7-4d57-9273-40a16496dcdd:/dev/nvme0n1p4]
Sep 10 14:11:56.874160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.874080    2576 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 14:11:56.879154 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.879038    2576 manager.go:216] Machine: {Timestamp:2026-09-10 14:11:56.877828745 +0000 UTC m=+0.315838801 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3093857 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22ba44efa822f76c8de5277f8d481c SystemUUID:ec22ba44-efa8-22f7-6c8d-e5277f8d481c BootID:8943df89-4ff1-46b7-972a-0b47bd094981 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:6a:3e:a5:f2:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:6a:3e:a5:f2:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:72:95:78:12:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 14:11:56.879154 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.879144    2576 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 14:11:56.879316 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.879263    2576 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 14:11:56.880193 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.880174    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:11:56.880861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.880838    2576 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 14:11:56.881010 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.880867    2576 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-41.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 14:11:56.881704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.881675    2576 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 14:11:56.881704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.881706    2576 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 14:11:56.881789 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.881714    2576 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:11:56.882334 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.882324    2576 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:11:56.883604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.883594    2576 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 14:11:56.887556 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887544    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:11:56.887671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887661    2576 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 14:11:56.887737 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887709    2576 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 14:11:56.887737 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887730    2576 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 14:11:56.887813 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887745    2576 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 14:11:56.887813 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887755    2576 kubelet.go:398] "Adding apiserver pod source"
Sep 10 14:11:56.887813 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.887763    2576 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 14:11:56.888133 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.888122    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:11:56.888163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.888142    2576 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:11:56.890378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.890354    2576 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 14:11:56.895287 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895269    2576 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 14:11:56.895382 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895314    2576 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 14:11:56.895559 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895544    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 14:11:56.895623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895576    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 14:11:56.895623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895590    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 14:11:56.895623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895600    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 14:11:56.895623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895620    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 14:11:56.895765 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.895628    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 14:11:56.896812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.896800    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 14:11:56.896871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.896814    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 14:11:56.896871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.896827    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 14:11:56.896871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.896846    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 14:11:56.896871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.896858    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 14:11:56.896871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.896871    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 14:11:56.897610 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.897599    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 14:11:56.897662 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.897613    2576 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 14:11:56.901948 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.901798    2576 server.go:1268] "Started kubelet"
Sep 10 14:11:56.901948 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.901922    2576 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 14:11:56.902070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.901988    2576 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 14:11:56.902639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.902607    2576 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 14:11:56.904329 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.904127    2576 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 14:11:56.904421 ip-10-0-128-41 systemd[1]: Started Kubernetes Kubelet.
Sep 10 14:11:56.905723 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.905696    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 14:11:56.905809 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.905762    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 14:11:56.905878 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.905848    2576 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-41.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 14:11:56.906600 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.906579    2576 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 14:11:56.909047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.909024    2576 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 14:11:56.909650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.909631    2576 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 14:11:56.909755 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.909733    2576 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 14:11:56.909901 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.909875    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:56.910487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910471    2576 factory.go:55] Registering systemd factory
Sep 10 14:11:56.910487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910487    2576 factory.go:223] Registration of the systemd container factory successfully
Sep 10 14:11:56.910742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910722    2576 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 14:11:56.910742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910737    2576 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 14:11:56.910855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910754    2576 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 14:11:56.910855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910753    2576 factory.go:153] Registering CRI-O factory
Sep 10 14:11:56.910855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910780    2576 factory.go:223] Registration of the crio container factory successfully
Sep 10 14:11:56.910855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910831    2576 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 14:11:56.910855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910846    2576 factory.go:103] Registering Raw factory
Sep 10 14:11:56.910855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910858    2576 manager.go:1196] Started watching for new ooms in manager
Sep 10 14:11:56.911075 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910922    2576 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 14:11:56.911075 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.910930    2576 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 14:11:56.911282 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.911271    2576 manager.go:349] Starting recovery of all containers
Sep 10 14:11:56.911556 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.910657    2576 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-41.ec2.internal.18d3fb0e8c833f47  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-41.ec2.internal,UID:ip-10-0-128-41.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-41.ec2.internal,},FirstTimestamp:2026-09-10 14:11:56.901744455 +0000 UTC m=+0.339754501,LastTimestamp:2026-09-10 14:11:56.901744455 +0000 UTC m=+0.339754501,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-41.ec2.internal,}"
Sep 10 14:11:56.920963 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.920932    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-845t9"
Sep 10 14:11:56.921826 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.921808    2576 manager.go:354] Recovery completed
Sep 10 14:11:56.922361 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.922337    2576 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-41.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 14:11:56.922472 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.922418    2576 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 14:11:56.926848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.926831    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-845t9"
Sep 10 14:11:56.927360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.927348    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:56.929392 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.929373    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:56.929459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.929408    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:56.929459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.929423    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:56.929994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.929980    2576 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 14:11:56.929994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.929992    2576 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 14:11:56.930107 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.930005    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:56.930107 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.930010    2576 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:11:56.931123 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.931105    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:56.931221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.931139    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:56.931221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.931154    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:56.931365 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.931350    2576 policy_none.go:49] "None policy: Start"
Sep 10 14:11:56.931422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.931370    2576 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 14:11:56.931422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.931382    2576 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 14:11:56.932522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.932508    2576 policy_none.go:47] "Start"
Sep 10 14:11:56.967293 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967271    2576 manager.go:337] "Starting Device Plugin manager"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.967316    2576 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967326    2576 server.go:87] "Starting device plugin registration server"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967746    2576 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967758    2576 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967853    2576 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967926    2576 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:56.967936    2576 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.968475    2576 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 14:11:56.975294 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:56.968518    2576 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.067917 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.067878    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:57.069632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.069598    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 14:11:57.070986 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.070960    2576 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 14:11:57.071089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.071023    2576 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 14:11:57.071089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.071051    2576 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 14:11:57.071165 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.071096    2576 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 14:11:57.072432 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.072409    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:57.072544 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.072453    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:57.072544 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.072472    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:57.072544 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.072510    2576 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.072974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.072869    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:57.080316 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.080249    2576 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.080316 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.080271    2576 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-41.ec2.internal\": node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.097061 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.097042    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.171598 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.171542    2576 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"]
Sep 10 14:11:57.171725 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.171649    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:57.173262 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.173245    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:57.173319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.173280    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:57.173319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.173295    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:57.174760 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.174746    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:57.174894 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.174877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.174931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.174909    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:57.175502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.175485    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:57.175577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.175515    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:57.175577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.175530    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:57.175671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.175581    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:57.175671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.175601    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:57.175671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.175609    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:57.176809 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.176796    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.176861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.176820    2576 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:11:57.177495 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.177481    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:11:57.177571 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.177503    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:11:57.177571 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.177514    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:11:57.197552 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.197531    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.206335 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.206313    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-41.ec2.internal\" not found" node="ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.210780 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.210765    2576 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-41.ec2.internal\" not found" node="ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.212832 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.212815    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b604e50155af53c6f49e2fae328d06fe-config\") pod \"kube-apiserver-proxy-ip-10-0-128-41.ec2.internal\" (UID: \"b604e50155af53c6f49e2fae328d06fe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.212882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.212843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b98ee08cd8fdb0bc6c64c3ab0503b412-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal\" (UID: \"b98ee08cd8fdb0bc6c64c3ab0503b412\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.212882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.212861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98ee08cd8fdb0bc6c64c3ab0503b412-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal\" (UID: \"b98ee08cd8fdb0bc6c64c3ab0503b412\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.298618 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.298588    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.313692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.313658    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98ee08cd8fdb0bc6c64c3ab0503b412-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal\" (UID: \"b98ee08cd8fdb0bc6c64c3ab0503b412\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.313795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.313708    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b604e50155af53c6f49e2fae328d06fe-config\") pod \"kube-apiserver-proxy-ip-10-0-128-41.ec2.internal\" (UID: \"b604e50155af53c6f49e2fae328d06fe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.313795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.313728    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b98ee08cd8fdb0bc6c64c3ab0503b412-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal\" (UID: \"b98ee08cd8fdb0bc6c64c3ab0503b412\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.313795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.313753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b98ee08cd8fdb0bc6c64c3ab0503b412-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal\" (UID: \"b98ee08cd8fdb0bc6c64c3ab0503b412\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.313795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.313779    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b98ee08cd8fdb0bc6c64c3ab0503b412-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal\" (UID: \"b98ee08cd8fdb0bc6c64c3ab0503b412\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.313930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.313825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b604e50155af53c6f49e2fae328d06fe-config\") pod \"kube-apiserver-proxy-ip-10-0-128-41.ec2.internal\" (UID: \"b604e50155af53c6f49e2fae328d06fe\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.399187 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.399091    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.499693 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.499652    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.509845 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.509821    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.514004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.513862    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:57.600124 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.600079    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.700748 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.700672    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.801155 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.801126    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.848692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.848652    2576 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 14:11:57.849126 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.848806    2576 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:11:57.902110 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:57.902083    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:57.928567 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.928533    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 14:06:58 +0000 UTC" deadline="2028-05-14 11:42:21.359516803 +0000 UTC"
Sep 10 14:11:57.928567 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:57.928566    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14685h30m23.430954169s"
Sep 10 14:11:58.002963 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:58.002892    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:58.027879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:58.027857    2576 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:58.035091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:58.035072    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:11:59.424685 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.424648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal" event={"ID":"b604e50155af53c6f49e2fae328d06fe","Type":"ContainerStarted","Data":"50da7b64c88eb6f7c68d4c2c8e47bbba28a6a22e62b854c6fd58e1d1ffb4e532"}
Sep 10 14:11:59.452806 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:59.452787    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:59.553377 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:59.553349    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:59.653935 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:59.653860    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:59.685185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.685161    2576 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:59.754456 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:11:59.754419    2576 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-41.ec2.internal\" not found"
Sep 10 14:11:59.754659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.754482    2576 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:11:59.759729 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.759710    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:59.770714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.770690    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:11:59.771697 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.771684    2576 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal"
Sep 10 14:11:59.781689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:11:59.781672    2576 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:12:00.189276 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:00.189224    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb98ee08cd8fdb0bc6c64c3ab0503b412.slice/crio-bda0cbf4e7e06a632c94d20e2283818b515692d0c3a8ebd56b8cf070f9840a55 WatchSource:0}: Error finding container bda0cbf4e7e06a632c94d20e2283818b515692d0c3a8ebd56b8cf070f9840a55: Status 404 returned error can't find the container with id bda0cbf4e7e06a632c94d20e2283818b515692d0c3a8ebd56b8cf070f9840a55
Sep 10 14:12:00.239005 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.238973    2576 apiserver.go:52] "Watching apiserver"
Sep 10 14:12:00.247019 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.246979    2576 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 14:12:00.248300 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.248255    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-57w2l","kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt","openshift-cluster-node-tuning-operator/tuned-22qmn","openshift-multus/network-metrics-daemon-sr27l","openshift-ovn-kubernetes/ovnkube-node-tnzpc","kube-system/global-pull-secret-syncer-qwlw4","openshift-dns/node-resolver-w4lmh","openshift-image-registry/node-ca-4spz6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal","openshift-multus/multus-additional-cni-plugins-hwkkb","openshift-multus/multus-qg85n","openshift-network-diagnostics/network-check-target-64znd","openshift-network-operator/iptables-alerter-xpc8v"]
Sep 10 14:12:00.250105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.250081    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.250195 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.250167    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:00.251302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.251284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:00.251378 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.251336    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:00.252491 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.252470    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.253754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.253736    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.255221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.255202    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.255329 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.255297    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:00.256634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.256616    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.256750 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.256734    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 14:12:00.257064 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.257046    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.257130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.257113    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.257170 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.257158    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6wc2\""
Sep 10 14:12:00.257952 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.257936    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.259340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.259256    2576 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 14:12:00.259408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.259338    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.260943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.260926    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.262569 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.262554    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.264143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.264124    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.265560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265540    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 14:12:00.265626 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265558    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265743    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265768    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265788    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265828    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265836    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.265953    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.266079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.266072    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267344    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267346    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-fltsx\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267429    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267459    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-5qvkw\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267474    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267500    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267504    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267432    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267511    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 14:12:00.267589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267430    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 14:12:00.268008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.267919    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6l56n\""
Sep 10 14:12:00.268061 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.268031    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cg8qz\""
Sep 10 14:12:00.268677 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.268658    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 14:12:00.268768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.268725    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xpdmt\""
Sep 10 14:12:00.268768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.268736    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-cm9gk\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.269496    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.269841    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.269880    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bjd7k\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.270089    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.270113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2rfm4\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.270141    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 14:12:00.271473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.270304    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 14:12:00.272541 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/652327f0-1aef-4f03-b450-d89fcacee365-dbus\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.272612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-var-lib-kubelet\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.272612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-host\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.272691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272617    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnnpn\" (UniqueName: \"kubernetes.io/projected/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-kube-api-access-fnnpn\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.272691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272643    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovnkube-config\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.272691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272670    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c701d1e-6488-480a-8555-6a8aadddef69-etc-tuned\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.272835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272703    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-cni-netd\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.272835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272755    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvrt9\" (UniqueName: \"kubernetes.io/projected/757c049b-a007-48ad-8a59-5d0a01af0bcb-kube-api-access-bvrt9\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.272835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272789    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/757c049b-a007-48ad-8a59-5d0a01af0bcb-iptables-alerter-script\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.272835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272813    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-sys\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.272991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272836    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysconfig\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.272991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-lib-modules\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.272991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqhdj\" (UniqueName: \"kubernetes.io/projected/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-kube-api-access-cqhdj\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.272991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272914    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/652327f0-1aef-4f03-b450-d89fcacee365-kubelet-config\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.272991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272939    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-kubernetes\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.272991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.272973    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-host\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-serviceca\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273054    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-run\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273076    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-systemd\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273098    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-registration-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgx7g\" (UniqueName: \"kubernetes.io/projected/8bc92705-262f-4580-bcc8-a81e39dd5dee-kube-api-access-sgx7g\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-node-log\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovn-node-metrics-cert\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273214    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-878v9\" (UniqueName: \"kubernetes.io/projected/8c701d1e-6488-480a-8555-6a8aadddef69-kube-api-access-878v9\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273292    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-run-netns\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273318    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-etc-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273371    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653fae07-2217-4639-89e2-465c3eea1bc7-konnectivity-ca\") pod \"konnectivity-agent-57w2l\" (UID: \"653fae07-2217-4639-89e2-465c3eea1bc7\") " pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273392    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c701d1e-6488-480a-8555-6a8aadddef69-tmp\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-device-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273443    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5st7v\" (UniqueName: \"kubernetes.io/projected/745510db-ad2d-4b22-97b7-4224f10153d5-kube-api-access-5st7v\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273469    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-ovn\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-log-socket\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273520    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-cni-bin\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.273553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-socket-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-slash\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273628    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273656    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-systemd\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273681    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-var-lib-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273699    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653fae07-2217-4639-89e2-465c3eea1bc7-agent-certs\") pod \"konnectivity-agent-57w2l\" (UID: \"653fae07-2217-4639-89e2-465c3eea1bc7\") " pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273732    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-run-ovn-kubernetes\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-modprobe-d\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273796    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysctl-d\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-kubelet\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273887    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273920    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-systemd-units\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.273963    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-env-overrides\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.274007    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysctl-conf\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.274630 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.274048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-sys-fs\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.274630 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.274080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovnkube-script-lib\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.274630 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.274100    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/757c049b-a007-48ad-8a59-5d0a01af0bcb-host-slash\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.280094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.280078    2576 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:12:00.312684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.312656    2576 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wtnf6"
Sep 10 14:12:00.322719 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.322691    2576 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wtnf6"
Sep 10 14:12:00.361895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.361867    2576 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 14:12:00.374543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374510    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-systemd\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.374543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-registration-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374568    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgx7g\" (UniqueName: \"kubernetes.io/projected/8bc92705-262f-4580-bcc8-a81e39dd5dee-kube-api-access-sgx7g\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374584    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-node-log\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374599    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovn-node-metrics-cert\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374622    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b327197-ec2d-4799-9164-bbea8925c918-tmp-dir\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374640    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-systemd\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-node-log\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374644    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-hostroot\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374646    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-registration-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374692    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-878v9\" (UniqueName: \"kubernetes.io/projected/8c701d1e-6488-480a-8555-6a8aadddef69-kube-api-access-878v9\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.374768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-run-netns\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374816    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-etc-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-run-netns\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374847    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-etc-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374873    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653fae07-2217-4639-89e2-465c3eea1bc7-konnectivity-ca\") pod \"konnectivity-agent-57w2l\" (UID: \"653fae07-2217-4639-89e2-465c3eea1bc7\") " pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374901    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-etc-kubernetes\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374910    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c701d1e-6488-480a-8555-6a8aadddef69-tmp\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-device-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.374975    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5st7v\" (UniqueName: \"kubernetes.io/projected/745510db-ad2d-4b22-97b7-4224f10153d5-kube-api-access-5st7v\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375000    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-device-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00231867-05ed-4f09-8495-092d8322939e-cni-binary-copy\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-ovn\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375173    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-socket-dir-parent\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375200    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-log-socket\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.375344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375223    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-cni-bin\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375250    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-ovn\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375282    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-log-socket\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375287    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-cni-bin\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375310    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt6vt\" (UniqueName: \"kubernetes.io/projected/7b327197-ec2d-4799-9164-bbea8925c918-kube-api-access-qt6vt\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375349    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6rh2\" (UniqueName: \"kubernetes.io/projected/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-kube-api-access-k6rh2\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375373    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-multus-certs\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375389    2576 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375396    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stb6c\" (UniqueName: \"kubernetes.io/projected/00231867-05ed-4f09-8495-092d8322939e-kube-api-access-stb6c\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375481    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-socket-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375517    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-slash\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375544    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375571    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/00231867-05ed-4f09-8495-092d8322939e-multus-daemon-config\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375595    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.375600    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375619    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-systemd\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-var-lib-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375675    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653fae07-2217-4639-89e2-465c3eea1bc7-agent-certs\") pod \"konnectivity-agent-57w2l\" (UID: \"653fae07-2217-4639-89e2-465c3eea1bc7\") " pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.375693    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:00.875670632 +0000 UTC m=+2.964128202 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375782    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-run-ovn-kubernetes\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-k8s-cni-cncf-io\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-modprobe-d\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysctl-d\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375889    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-kubelet\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375916    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375941    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.375990    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-systemd-units\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376013    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-env-overrides\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b327197-ec2d-4799-9164-bbea8925c918-hosts-file\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376061    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cnibin\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.376867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-cni-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysctl-conf\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376120    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/653fae07-2217-4639-89e2-465c3eea1bc7-konnectivity-ca\") pod \"konnectivity-agent-57w2l\" (UID: \"653fae07-2217-4639-89e2-465c3eea1bc7\") " pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376133    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-sys-fs\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-sys-fs\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-run-ovn-kubernetes\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovnkube-script-lib\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376280    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-slash\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376319    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-kubelet\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376349    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-socket-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376394    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376411    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysctl-d\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376253    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-systemd-units\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-systemd\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376143    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-modprobe-d\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376510    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8bc92705-262f-4580-bcc8-a81e39dd5dee-etc-selinux\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/757c049b-a007-48ad-8a59-5d0a01af0bcb-host-slash\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376559    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-var-lib-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377409 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.376582    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-cnibin\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.376651    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:00.876626692 +0000 UTC m=+2.965084241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/757c049b-a007-48ad-8a59-5d0a01af0bcb-host-slash\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376680    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysctl-conf\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376702    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/652327f0-1aef-4f03-b450-d89fcacee365-dbus\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376732    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-var-lib-kubelet\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-host\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376801    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-var-lib-kubelet\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnnpn\" (UniqueName: \"kubernetes.io/projected/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-kube-api-access-fnnpn\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376824    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-env-overrides\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376831    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/652327f0-1aef-4f03-b450-d89fcacee365-dbus\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376867    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-host\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376926    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovnkube-config\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.376959    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-netns\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-cni-bin\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377189    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-cni-multus\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.377944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377439    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-conf-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377462    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovnkube-script-lib\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c701d1e-6488-480a-8555-6a8aadddef69-etc-tuned\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377527    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovnkube-config\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-cni-netd\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377586    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvrt9\" (UniqueName: \"kubernetes.io/projected/757c049b-a007-48ad-8a59-5d0a01af0bcb-kube-api-access-bvrt9\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377640    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/757c049b-a007-48ad-8a59-5d0a01af0bcb-iptables-alerter-script\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-sys\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377684    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysconfig\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-lib-modules\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377742    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqhdj\" (UniqueName: \"kubernetes.io/projected/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-kube-api-access-cqhdj\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377818    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-system-cni-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377850    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-kubelet\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377879    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/652327f0-1aef-4f03-b450-d89fcacee365-kubelet-config\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377937    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-kubernetes\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.377963    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-host\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378012    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-serviceca\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.378914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-system-cni-dir\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378180    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-run\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378301    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-run\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378351    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/757c049b-a007-48ad-8a59-5d0a01af0bcb-iptables-alerter-script\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378415    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-sys\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378475    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-host\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378541    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/652327f0-1aef-4f03-b450-d89fcacee365-kubelet-config\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-kubernetes\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-host-cni-netd\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378734    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-run-openvswitch\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-serviceca\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-lib-modules\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.379741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.378930    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/8c701d1e-6488-480a-8555-6a8aadddef69-etc-sysconfig\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.380278 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.380082    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-ovn-node-metrics-cert\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.380278 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.380136    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/653fae07-2217-4639-89e2-465c3eea1bc7-agent-certs\") pod \"konnectivity-agent-57w2l\" (UID: \"653fae07-2217-4639-89e2-465c3eea1bc7\") " pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.380965 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.380942    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8c701d1e-6488-480a-8555-6a8aadddef69-tmp\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.381150 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.381135    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/8c701d1e-6488-480a-8555-6a8aadddef69-etc-tuned\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.384219 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.384200    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-878v9\" (UniqueName: \"kubernetes.io/projected/8c701d1e-6488-480a-8555-6a8aadddef69-kube-api-access-878v9\") pod \"tuned-22qmn\" (UID: \"8c701d1e-6488-480a-8555-6a8aadddef69\") " pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.385778 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.385757    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5st7v\" (UniqueName: \"kubernetes.io/projected/745510db-ad2d-4b22-97b7-4224f10153d5-kube-api-access-5st7v\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.385987 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.385968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgx7g\" (UniqueName: \"kubernetes.io/projected/8bc92705-262f-4580-bcc8-a81e39dd5dee-kube-api-access-sgx7g\") pod \"aws-ebs-csi-driver-node-qqcdt\" (UID: \"8bc92705-262f-4580-bcc8-a81e39dd5dee\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.392869 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.390454    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:00.392869 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.390487    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:00.392869 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.390501    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:00.392869 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.390721    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:00.890688182 +0000 UTC m=+2.979145746 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:00.392869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.391923    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqhdj\" (UniqueName: \"kubernetes.io/projected/7f0a113a-476d-48e3-b4c4-5b4db4b11f54-kube-api-access-cqhdj\") pod \"ovnkube-node-tnzpc\" (UID: \"7f0a113a-476d-48e3-b4c4-5b4db4b11f54\") " pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.392869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.392700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnnpn\" (UniqueName: \"kubernetes.io/projected/c22a5c55-fbd5-4a43-bf63-0ec6b132b0be-kube-api-access-fnnpn\") pod \"node-ca-4spz6\" (UID: \"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be\") " pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.393748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.393726    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvrt9\" (UniqueName: \"kubernetes.io/projected/757c049b-a007-48ad-8a59-5d0a01af0bcb-kube-api-access-bvrt9\") pod \"iptables-alerter-xpc8v\" (UID: \"757c049b-a007-48ad-8a59-5d0a01af0bcb\") " pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.426875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.426843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal" event={"ID":"b98ee08cd8fdb0bc6c64c3ab0503b412","Type":"ContainerStarted","Data":"bda0cbf4e7e06a632c94d20e2283818b515692d0c3a8ebd56b8cf070f9840a55"}
Sep 10 14:12:00.475800 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.475723    2576 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:12:00.479056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479022    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b327197-ec2d-4799-9164-bbea8925c918-tmp-dir\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.479207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-hostroot\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479091    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-etc-kubernetes\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479116    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.479207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00231867-05ed-4f09-8495-092d8322939e-cni-binary-copy\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-etc-kubernetes\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479264    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-hostroot\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-socket-dir-parent\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479362    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt6vt\" (UniqueName: \"kubernetes.io/projected/7b327197-ec2d-4799-9164-bbea8925c918-kube-api-access-qt6vt\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479385    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6rh2\" (UniqueName: \"kubernetes.io/projected/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-kube-api-access-k6rh2\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479413    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-multus-certs\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479432    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stb6c\" (UniqueName: \"kubernetes.io/projected/00231867-05ed-4f09-8495-092d8322939e-kube-api-access-stb6c\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479436    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b327197-ec2d-4799-9164-bbea8925c918-tmp-dir\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479494    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479525    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/00231867-05ed-4f09-8495-092d8322939e-multus-daemon-config\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479554    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479604    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-k8s-cni-cncf-io\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b327197-ec2d-4799-9164-bbea8925c918-hosts-file\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479688    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cnibin\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-cni-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-cnibin\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-netns\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.479812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479796    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-cni-bin\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479831    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-cni-multus\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-conf-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479890    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-system-cni-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479914    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-kubelet\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.479945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-system-cni-dir\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480056    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-system-cni-dir\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.480325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00231867-05ed-4f09-8495-092d8322939e-cni-binary-copy\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.480694 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480608    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.480694 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480628    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.480694 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480688    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-socket-dir-parent\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480894    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-multus-certs\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480919    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-k8s-cni-cncf-io\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-cnibin\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480950    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b327197-ec2d-4799-9164-bbea8925c918-hosts-file\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-cni-bin\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480984    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-cni-multus\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.480988    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-run-netns\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481023    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-cni-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481061    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-system-cni-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481067    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-multus-conf-dir\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481068    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481083    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/00231867-05ed-4f09-8495-092d8322939e-multus-daemon-config\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481095    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/00231867-05ed-4f09-8495-092d8322939e-host-var-lib-kubelet\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.481198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481178    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cnibin\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.481714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.481325    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-cni-binary-copy\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.488551 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.488529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stb6c\" (UniqueName: \"kubernetes.io/projected/00231867-05ed-4f09-8495-092d8322939e-kube-api-access-stb6c\") pod \"multus-qg85n\" (UID: \"00231867-05ed-4f09-8495-092d8322939e\") " pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.491782 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.491764    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt6vt\" (UniqueName: \"kubernetes.io/projected/7b327197-ec2d-4799-9164-bbea8925c918-kube-api-access-qt6vt\") pod \"node-resolver-w4lmh\" (UID: \"7b327197-ec2d-4799-9164-bbea8925c918\") " pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.492729 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.492704    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6rh2\" (UniqueName: \"kubernetes.io/projected/9b6fcde7-26f5-4234-a09a-f762fcdac8e4-kube-api-access-k6rh2\") pod \"multus-additional-cni-plugins-hwkkb\" (UID: \"9b6fcde7-26f5-4234-a09a-f762fcdac8e4\") " pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.563354 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.563314    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt"
Sep 10 14:12:00.569644 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.569622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-22qmn"
Sep 10 14:12:00.578479 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.578449    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:00.583377 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.583348    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4spz6"
Sep 10 14:12:00.589047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.589024    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:00.595801 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.595779    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xpc8v"
Sep 10 14:12:00.602511 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.602490    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4lmh"
Sep 10 14:12:00.609409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.609387    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hwkkb"
Sep 10 14:12:00.615371 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.615345    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qg85n"
Sep 10 14:12:00.883099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.883050    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:00.883274 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.883142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:00.883274 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.883200    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:00.883408 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.883289    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:01.883267183 +0000 UTC m=+3.971724747 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:00.883408 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.883303    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:00.883408 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.883351    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:01.883333349 +0000 UTC m=+3.971790902 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:00.983626 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:00.983594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:00.983798 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.983726    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:00.983798 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.983741    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:00.983798 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.983749    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:00.983935 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:00.983801    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:01.983787311 +0000 UTC m=+4.072244860 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:01.057135 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:01.057099    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b327197_ec2d_4799_9164_bbea8925c918.slice/crio-d5adcec74029f2de646408cfab89214283ad163211eb01f4b54af0c18a010fba WatchSource:0}: Error finding container d5adcec74029f2de646408cfab89214283ad163211eb01f4b54af0c18a010fba: Status 404 returned error can't find the container with id d5adcec74029f2de646408cfab89214283ad163211eb01f4b54af0c18a010fba
Sep 10 14:12:01.057784 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:01.057762    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00231867_05ed_4f09_8495_092d8322939e.slice/crio-2f5c27f7ad0f27be4fd187564044dfe5751d585c8638d3c9d5ad1362bf774fa0 WatchSource:0}: Error finding container 2f5c27f7ad0f27be4fd187564044dfe5751d585c8638d3c9d5ad1362bf774fa0: Status 404 returned error can't find the container with id 2f5c27f7ad0f27be4fd187564044dfe5751d585c8638d3c9d5ad1362bf774fa0
Sep 10 14:12:01.070941 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:01.070922    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bc92705_262f_4580_bcc8_a81e39dd5dee.slice/crio-1fd9e4d67008edeb60162c7f2aa383a1de70a743ef5ac684bdb24f9bbe03c1d0 WatchSource:0}: Error finding container 1fd9e4d67008edeb60162c7f2aa383a1de70a743ef5ac684bdb24f9bbe03c1d0: Status 404 returned error can't find the container with id 1fd9e4d67008edeb60162c7f2aa383a1de70a743ef5ac684bdb24f9bbe03c1d0
Sep 10 14:12:01.074415 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:01.074376    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b6fcde7_26f5_4234_a09a_f762fcdac8e4.slice/crio-248e67b6f97ff52df5b407058f7b28d506bb2dbd82f76dd2f123972e7103a920 WatchSource:0}: Error finding container 248e67b6f97ff52df5b407058f7b28d506bb2dbd82f76dd2f123972e7103a920: Status 404 returned error can't find the container with id 248e67b6f97ff52df5b407058f7b28d506bb2dbd82f76dd2f123972e7103a920
Sep 10 14:12:01.074953 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:01.074932    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod653fae07_2217_4639_89e2_465c3eea1bc7.slice/crio-7fc5fd3f69bea44e340385265ad913a15cd2215b50f03050358a03eae9cedf70 WatchSource:0}: Error finding container 7fc5fd3f69bea44e340385265ad913a15cd2215b50f03050358a03eae9cedf70: Status 404 returned error can't find the container with id 7fc5fd3f69bea44e340385265ad913a15cd2215b50f03050358a03eae9cedf70
Sep 10 14:12:01.325148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.325086    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:07:00 +0000 UTC" deadline="2028-02-08 05:14:05.219988754 +0000 UTC"
Sep 10 14:12:01.325148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.325138    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12375h2m3.894853375s"
Sep 10 14:12:01.430478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.430435    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4lmh" event={"ID":"7b327197-ec2d-4799-9164-bbea8925c918","Type":"ContainerStarted","Data":"d5adcec74029f2de646408cfab89214283ad163211eb01f4b54af0c18a010fba"}
Sep 10 14:12:01.434333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.434302    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal" event={"ID":"b604e50155af53c6f49e2fae328d06fe","Type":"ContainerStarted","Data":"d13f42697bf84ab0ac30a4ad720a19814e30af28ef22f46d6283f375b967c9ed"}
Sep 10 14:12:01.435715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.435686    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-22qmn" event={"ID":"8c701d1e-6488-480a-8555-6a8aadddef69","Type":"ContainerStarted","Data":"971bc88e34fe34f29003656d94108c9a1d43db5f665fa0273cd091aa381585ef"}
Sep 10 14:12:01.436879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.436839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4spz6" event={"ID":"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be","Type":"ContainerStarted","Data":"5d078c5a813c7bad523796275861477b5d9948e7a21eb7ea65b7d96fc2881be8"}
Sep 10 14:12:01.438023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.438000    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"fc58e210b8d9f2e2caf13049786d315155f6b3fbe5de058e2186a959f716b204"}
Sep 10 14:12:01.439293 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.439261    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-57w2l" event={"ID":"653fae07-2217-4639-89e2-465c3eea1bc7","Type":"ContainerStarted","Data":"7fc5fd3f69bea44e340385265ad913a15cd2215b50f03050358a03eae9cedf70"}
Sep 10 14:12:01.440489 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.440470    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt" event={"ID":"8bc92705-262f-4580-bcc8-a81e39dd5dee","Type":"ContainerStarted","Data":"1fd9e4d67008edeb60162c7f2aa383a1de70a743ef5ac684bdb24f9bbe03c1d0"}
Sep 10 14:12:01.441635 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.441615    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xpc8v" event={"ID":"757c049b-a007-48ad-8a59-5d0a01af0bcb","Type":"ContainerStarted","Data":"61ebea49db1ebb694439353678d60594ca3a3c9cc2f3df6dba583484d7779bf6"}
Sep 10 14:12:01.443128 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.443111    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerStarted","Data":"248e67b6f97ff52df5b407058f7b28d506bb2dbd82f76dd2f123972e7103a920"}
Sep 10 14:12:01.444982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.444961    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qg85n" event={"ID":"00231867-05ed-4f09-8495-092d8322939e","Type":"ContainerStarted","Data":"2f5c27f7ad0f27be4fd187564044dfe5751d585c8638d3c9d5ad1362bf774fa0"}
Sep 10 14:12:01.449774 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.449736    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-41.ec2.internal" podStartSLOduration=2.449724973 podStartE2EDuration="2.449724973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:01.449634061 +0000 UTC m=+3.538091633" watchObservedRunningTime="2026-09-10 14:12:01.449724973 +0000 UTC m=+3.538182543"
Sep 10 14:12:01.889466 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.889430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:01.889643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.889521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:01.889643 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.889575    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:01.889757 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.889647    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:03.889624906 +0000 UTC m=+5.978082469 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:01.889757 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.889657    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:01.889757 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.889711    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:03.889694512 +0000 UTC m=+5.978152083 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:01.989931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:01.989895    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:01.990091 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.990065    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:01.990167 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.990093    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:01.990167 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.990106    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:01.990302 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:01.990172    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:03.990152625 +0000 UTC m=+6.078610182 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:02.326221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.326127    2576 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:07:00 +0000 UTC" deadline="2028-02-10 21:34:49.685884934 +0000 UTC"
Sep 10 14:12:02.326221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.326172    2576 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12439h22m47.359718407s"
Sep 10 14:12:02.423744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.421604    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:02.423744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.421667    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:02.423744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.421672    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:02.423744 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:02.421774    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:02.423744 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:02.421869    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:02.423744 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:02.421967    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:02.460991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.459936    2576 generic.go:356] "Generic (PLEG): container finished" podID="b98ee08cd8fdb0bc6c64c3ab0503b412" containerID="365af6fba89593c105728d0902ead61b7a09d7bf6a509ce647e2d36221490af4" exitCode=0
Sep 10 14:12:02.460991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:02.460906    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal" event={"ID":"b98ee08cd8fdb0bc6c64c3ab0503b412","Type":"ContainerDied","Data":"365af6fba89593c105728d0902ead61b7a09d7bf6a509ce647e2d36221490af4"}
Sep 10 14:12:03.489733 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:03.489470    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal" event={"ID":"b98ee08cd8fdb0bc6c64c3ab0503b412","Type":"ContainerStarted","Data":"0803294154c098feb0c4d0046664fd734499a4c0994730ce4d55a9da64498e48"}
Sep 10 14:12:03.908470 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:03.908382    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:03.908636 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:03.908468    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:03.908636 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:03.908600    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:03.908737 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:03.908665    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:07.908645245 +0000 UTC m=+9.997102799 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:03.909120 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:03.909100    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:03.909200 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:03.909157    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:07.909141494 +0000 UTC m=+9.997599047 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:04.010265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:04.009795    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:04.010265 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.009985    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:04.010265 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.010003    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:04.010265 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.010017    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:04.010265 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.010074    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:08.010055819 +0000 UTC m=+10.098513373 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:04.423085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:04.422366    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:04.423085 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.422497    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:04.423085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:04.422914    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:04.423085 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.423023    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:04.423598 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:04.423567    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:04.423737 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:04.423671    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:06.423102 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:06.423054    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:06.423478 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:06.423184    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:06.423575 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:06.423552    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:06.423684 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:06.423653    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:06.423743 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:06.423696    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:06.423837 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:06.423815    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:07.942364 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:07.942330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:07.942756 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:07.942384    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:07.942756 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:07.942507    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:07.942756 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:07.942513    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:07.942756 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:07.942567    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:15.942550018 +0000 UTC m=+18.031007569 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:07.942756 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:07.942581    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:15.942575411 +0000 UTC m=+18.031032960 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:08.043500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:08.043453    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:08.043657 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.043640    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:08.043721 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.043666    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:08.043721 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.043681    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:08.043814 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.043746    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:16.04372532 +0000 UTC m=+18.132182884 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:08.423111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:08.422145    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:08.423111 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.422348    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:08.423111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:08.422755    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:08.423111 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.422848    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:08.423111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:08.422910    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:08.423111 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:08.422984    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:10.421364 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:10.421330    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:10.421774 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:10.421457    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:10.421774 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:10.421343    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:10.421774 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:10.421583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:10.421774 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:10.421650    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:10.421774 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:10.421704    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:12.421514 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:12.421477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:12.421966 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:12.421475    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:12.421966 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:12.421601    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:12.421966 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:12.421681    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:12.421966 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:12.421477    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:12.421966 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:12.421776    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:14.421181 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:14.421148    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:14.421684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:14.421150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:14.421684 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:14.421278    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:14.421684 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:14.421358    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:14.421684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:14.421150    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:14.421684 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:14.421436    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:16.003307 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:16.003212    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:16.003307 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:16.003294    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:16.003800 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.003358    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:16.003800 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.003419    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:16.003800 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.003449    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:32.003424467 +0000 UTC m=+34.091882043 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:16.003800 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.003471    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:32.003457906 +0000 UTC m=+34.091915456 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:16.104008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:16.103962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:16.104188 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.104153    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:16.104188 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.104181    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:16.104293 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.104195    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:16.104293 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.104278    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:32.104257514 +0000 UTC m=+34.192715087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:16.421655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:16.421620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:16.421655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:16.421658    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:16.421868 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:16.421620    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:16.421868 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.421753    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:16.421868 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.421837    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:16.422022 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:16.421911    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:18.421370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:18.421326    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:18.421746 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:18.421458    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:18.421853 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:18.421826    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:18.421983 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:18.421925    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:18.422109 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:18.422087    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:18.422205 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:18.422183    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:19.525900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.525397    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-22qmn" event={"ID":"8c701d1e-6488-480a-8555-6a8aadddef69","Type":"ContainerStarted","Data":"ded3ea87eb7338ba85afcafb626351cf9f02bfaadf3f2f9322326848178b0ead"}
Sep 10 14:12:19.529886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.529131    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4spz6" event={"ID":"c22a5c55-fbd5-4a43-bf63-0ec6b132b0be","Type":"ContainerStarted","Data":"68bd8ef72d2d1a075071dec413eb70902d130cef6d7bb9aaf3c9bc6508a27b9d"}
Sep 10 14:12:19.532038 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.531509    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"0b2d1eeca6dc8260922066b429b60fc797de1a7027f13b61fbec65273e519ba7"}
Sep 10 14:12:19.533079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.533054    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-57w2l" event={"ID":"653fae07-2217-4639-89e2-465c3eea1bc7","Type":"ContainerStarted","Data":"c321a3f6e349fbd0758fa40f1f97302d70a81efbeaf6a0703282fc2705e811ff"}
Sep 10 14:12:19.535571 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.535542    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt" event={"ID":"8bc92705-262f-4580-bcc8-a81e39dd5dee","Type":"ContainerStarted","Data":"781fef0be3f666abcdcca148af4ff8df954c83afec6253a8c7ca63acd23beae0"}
Sep 10 14:12:19.537513 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.537155    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerStarted","Data":"e99a59659894ccb95680df391b017d1cbde957d1cbc9d67075022785a5467421"}
Sep 10 14:12:19.539712 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.539688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qg85n" event={"ID":"00231867-05ed-4f09-8495-092d8322939e","Type":"ContainerStarted","Data":"a00415981421cd8c58a65bda65f178b01b9f296ff81cac1da8687acffc5a8e1c"}
Sep 10 14:12:19.542069 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.542041    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4lmh" event={"ID":"7b327197-ec2d-4799-9164-bbea8925c918","Type":"ContainerStarted","Data":"e9183bcab11973610b1d3a0a46c7e61097716940a525f7703a8fded4f0d82d93"}
Sep 10 14:12:19.546030 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.545993    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-41.ec2.internal" podStartSLOduration=20.545980884 podStartE2EDuration="20.545980884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:03.509525991 +0000 UTC m=+5.597983566" watchObservedRunningTime="2026-09-10 14:12:19.545980884 +0000 UTC m=+21.634438455"
Sep 10 14:12:19.546462 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.546426    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-22qmn" podStartSLOduration=3.353256098 podStartE2EDuration="21.546412274s" totalImagesPullingTime="18.193156176s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:19.545810492 +0000 UTC m=+21.634268076" watchObservedRunningTime="2026-09-10 14:12:19.546412274 +0000 UTC m=+21.634869846"
Sep 10 14:12:19.591222 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.591150    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qg85n" podStartSLOduration=3.352136509 podStartE2EDuration="21.591134192s" totalImagesPullingTime="18.238997683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:19.5910017 +0000 UTC m=+21.679459271" watchObservedRunningTime="2026-09-10 14:12:19.591134192 +0000 UTC m=+21.679591763"
Sep 10 14:12:19.620300 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.620254    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4spz6" podStartSLOduration=3.450398341 podStartE2EDuration="21.620223525s" totalImagesPullingTime="18.169825184s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:19.6061919 +0000 UTC m=+21.694649470" watchObservedRunningTime="2026-09-10 14:12:19.620223525 +0000 UTC m=+21.708681096"
Sep 10 14:12:19.620593 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.620572    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-57w2l" podStartSLOduration=3.44647246 podStartE2EDuration="21.620562836s" totalImagesPullingTime="18.174090376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:19.620051526 +0000 UTC m=+21.708509096" watchObservedRunningTime="2026-09-10 14:12:19.620562836 +0000 UTC m=+21.709020408"
Sep 10 14:12:19.750753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.750721    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:19.821931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:19.821905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:20.423958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.423925    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:20.424144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.423926    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:20.424144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.424046    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:20.424144 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:20.424043    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:20.424331 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:20.424153    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:20.424331 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:20.424218    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:20.547684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.547589    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"688cdea47ad9fd348a47518807e14d7b080174a71f5d3bfbcbfb76e4a9afbfd5"}
Sep 10 14:12:20.547684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.547668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"5eacc35c31577887fcaf05d22141bd30050d805a1140a6f1ffaccaa1ab70e20b"}
Sep 10 14:12:20.548497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.547700    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"fe200195af36088e5d15a6ed46cfac090e22d0d0e51075273f013224076d83ad"}
Sep 10 14:12:20.548497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.547741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"e4b2cb28f4c90e2fc6e8818abf5f36f1727305c18aa8b75a75197c5552e6918a"}
Sep 10 14:12:20.548497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.547776    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"8c55eba55e87946ad6282f35958ce3c5944aa72f45909afcd0f175077311d284"}
Sep 10 14:12:20.549070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.549047    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xpc8v" event={"ID":"757c049b-a007-48ad-8a59-5d0a01af0bcb","Type":"ContainerStarted","Data":"f57c6aec87b9f8e37f3b762e9af4d1673f931873c69bbef0e528d953ba35d013"}
Sep 10 14:12:20.550493 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.550471    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6fcde7-26f5-4234-a09a-f762fcdac8e4" containerID="e99a59659894ccb95680df391b017d1cbde957d1cbc9d67075022785a5467421" exitCode=0
Sep 10 14:12:20.551141 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.551124    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerDied","Data":"e99a59659894ccb95680df391b017d1cbde957d1cbc9d67075022785a5467421"}
Sep 10 14:12:20.565280 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.565213    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w4lmh" podStartSLOduration=4.382386536 podStartE2EDuration="22.565201947s" totalImagesPullingTime="18.182815411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:19.640201667 +0000 UTC m=+21.728659242" watchObservedRunningTime="2026-09-10 14:12:20.565201947 +0000 UTC m=+22.653659518"
Sep 10 14:12:20.565608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.565579    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xpc8v" podStartSLOduration=4.3912685719999995 podStartE2EDuration="22.565569844s" totalImagesPullingTime="18.174301272s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:20.56492329 +0000 UTC m=+22.653380861" watchObservedRunningTime="2026-09-10 14:12:20.565569844 +0000 UTC m=+22.654027472"
Sep 10 14:12:20.684033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.684002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:20.788125 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:20.788086    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:21.252418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:21.252389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:21.435705 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:21.435555    2576 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 14:12:21.555523 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:21.555412    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt" event={"ID":"8bc92705-262f-4580-bcc8-a81e39dd5dee","Type":"ContainerStarted","Data":"f0f1a701502b1f31c7ee810ad0d0094dc504ee3ff4d61103d4b49faf2f988612"}
Sep 10 14:12:21.963100 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:21.963039    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:22.337540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.337399    2576 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T14:12:21.435702806Z","UUID":"a62ee90c-ab06-43c9-b766-c0c05a9ba192","Handler":null,"Name":"","Endpoint":""}
Sep 10 14:12:22.339249 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.339212    2576 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 14:12:22.339374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.339271    2576 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 14:12:22.434401 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.434361    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:22.434561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.434372    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:22.434561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.434495    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:22.434561 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:22.434480    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:22.434741 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:22.434587    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:22.434741 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:22.434686    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:22.561332 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.561289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"63c4e90268ec7061ceb6fadfd97171f32393113c02d2040c649f7d768672010e"}
Sep 10 14:12:22.626835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.626756    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:22.627409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:22.627385    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:23.565909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:23.565877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt" event={"ID":"8bc92705-262f-4580-bcc8-a81e39dd5dee","Type":"ContainerStarted","Data":"fbf0f5a8b725e568c0c799707898103f7eb2994b9beaded065f0b6c7fb48ecfd"}
Sep 10 14:12:23.566597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:23.566578    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-57w2l"
Sep 10 14:12:23.589622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:23.589573    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qqcdt" podStartSLOduration=4.359954072 podStartE2EDuration="25.589523124s" totalImagesPullingTime="21.229569052s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:23.589365716 +0000 UTC m=+25.677823288" watchObservedRunningTime="2026-09-10 14:12:23.589523124 +0000 UTC m=+25.677980740"
Sep 10 14:12:23.654424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:23.654397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:24.423262 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.421576    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:24.423262 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.421585    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:24.423262 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:24.421947    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:24.423262 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:24.422014    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:24.423262 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.421622    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:24.423262 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:24.422091    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:24.573437 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.573352    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" event={"ID":"7f0a113a-476d-48e3-b4c4-5b4db4b11f54","Type":"ContainerStarted","Data":"78b7d00456c265d30b39ab3ffeb85652ab9ebcf9242e41b07ff84aae1168e660"}
Sep 10 14:12:24.603214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.603156    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc" podStartSLOduration=8.350152109 podStartE2EDuration="26.603139265s" totalImagesPullingTime="18.252987156s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:24.601459348 +0000 UTC m=+26.689916950" watchObservedRunningTime="2026-09-10 14:12:24.603139265 +0000 UTC m=+26.691596843"
Sep 10 14:12:24.730313 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.730281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:24.827219 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.827112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:24.927907 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:24.927289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:25.575534 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:25.575509    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 14:12:25.575980 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:25.575961    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:25.576250 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:25.576221    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:25.595789 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:25.595755    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:25.595957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:25.595879    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:26.145643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.145609    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-64znd"]
Sep 10 14:12:26.145793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.145752    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:26.145890 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:26.145865    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:26.148108 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.148085    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qwlw4"]
Sep 10 14:12:26.148265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.148191    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:26.148320 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:26.148282    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:26.148935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.148908    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sr27l"]
Sep 10 14:12:26.149070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.149022    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:26.149142 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:26.149108    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:26.579369 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.579340    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6fcde7-26f5-4234-a09a-f762fcdac8e4" containerID="bc9ccb56c46cf73b9e5a18983a7c5ea34723fb0f4dfcd3499bde00871215a1af" exitCode=0
Sep 10 14:12:26.580093 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.579433    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerDied","Data":"bc9ccb56c46cf73b9e5a18983a7c5ea34723fb0f4dfcd3499bde00871215a1af"}
Sep 10 14:12:26.580093 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:26.579472    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 14:12:27.582221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:27.582183    2576 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 14:12:28.424522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:28.424334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:28.424676 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:28.424334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:28.424676 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:28.424612    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:28.424676 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:28.424334    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:28.424676 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:28.424655    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:28.424896 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:28.424717    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:28.586553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:28.586457    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6fcde7-26f5-4234-a09a-f762fcdac8e4" containerID="2a6abaf2e02ef04914847bc1a6b9841decffb870a6a3f3537e8c7867e6e74f56" exitCode=0
Sep 10 14:12:28.586553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:28.586539    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerDied","Data":"2a6abaf2e02ef04914847bc1a6b9841decffb870a6a3f3537e8c7867e6e74f56"}
Sep 10 14:12:28.934985 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:28.934942    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:29.107249 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:29.107215    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:29.208165 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:29.208077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:30.420912 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:30.420877    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:30.421328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:30.420893    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:30.421328 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:30.420990    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-64znd" podUID="59a86a34-c66c-4e51-95d6-4632f72f0454"
Sep 10 14:12:30.421328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:30.421006    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:30.421328 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:30.421094    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sr27l" podUID="745510db-ad2d-4b22-97b7-4224f10153d5"
Sep 10 14:12:30.421328 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:30.421170    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qwlw4" podUID="652327f0-1aef-4f03-b450-d89fcacee365"
Sep 10 14:12:30.594548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:30.594476    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6fcde7-26f5-4234-a09a-f762fcdac8e4" containerID="deac2ce17dd9c2f536c87c25d396b1fa5e3c2d588d7e58ceec0b8db93fcb818d" exitCode=0
Sep 10 14:12:30.594548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:30.594516    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerDied","Data":"deac2ce17dd9c2f536c87c25d396b1fa5e3c2d588d7e58ceec0b8db93fcb818d"}
Sep 10 14:12:32.024464 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.024418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:32.024900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.024486    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:32.024900 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.024573    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:32.024900 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.024576    2576 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:32.024900 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.024636    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:04.024618208 +0000 UTC m=+66.113075774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:12:32.024900 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.024651    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret podName:652327f0-1aef-4f03-b450-d89fcacee365 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:04.024645463 +0000 UTC m=+66.113103013 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret") pod "global-pull-secret-syncer-qwlw4" (UID: "652327f0-1aef-4f03-b450-d89fcacee365") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:12:32.125214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.125128    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:32.125389 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.125324    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:12:32.125389 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.125340    2576 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:12:32.125389 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.125350    2576 projected.go:196] Error preparing data for projected volume kube-api-access-xrnc9 for pod openshift-network-diagnostics/network-check-target-64znd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:32.125505 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.125401    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9 podName:59a86a34-c66c-4e51-95d6-4632f72f0454 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:04.125387311 +0000 UTC m=+66.213844859 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xrnc9" (UniqueName: "kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9") pod "network-check-target-64znd" (UID: "59a86a34-c66c-4e51-95d6-4632f72f0454") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:12:32.195445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.195421    2576 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-41.ec2.internal" event="NodeReady"
Sep 10 14:12:32.195634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.195564    2576 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 14:12:32.266413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.266379    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bxrf8"]
Sep 10 14:12:32.268516 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.268486    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.269538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.269514    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-xs8p9"]
Sep 10 14:12:32.271139 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.271120    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.272099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.272035    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 14:12:32.272099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.272068    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 14:12:32.272460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.272444    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwswr\""
Sep 10 14:12:32.274258 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.273634    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 14:12:32.274258 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.273730    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 14:12:32.274258 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.273853    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 14:12:32.274258 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.273971    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4qtdq\""
Sep 10 14:12:32.283116 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.283095    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xs8p9"]
Sep 10 14:12:32.288617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.288591    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bxrf8"]
Sep 10 14:12:32.326577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.326545    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/da042357-c42d-4a96-970c-1ad9e2bbee80-config-volume\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.326577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.326576    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da042357-c42d-4a96-970c-1ad9e2bbee80-tmp-dir\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.326748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.326597    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456e19b-d821-4774-9c39-ce6bc666bc6d-cert\") pod \"ingress-canary-xs8p9\" (UID: \"f456e19b-d821-4774-9c39-ce6bc666bc6d\") " pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.326748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.326650    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlvt5\" (UniqueName: \"kubernetes.io/projected/da042357-c42d-4a96-970c-1ad9e2bbee80-kube-api-access-dlvt5\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.326748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.326706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.326748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.326726    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xrqn\" (UniqueName: \"kubernetes.io/projected/f456e19b-d821-4774-9c39-ce6bc666bc6d-kube-api-access-6xrqn\") pod \"ingress-canary-xs8p9\" (UID: \"f456e19b-d821-4774-9c39-ce6bc666bc6d\") " pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.421123 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.421092    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:12:32.421288 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.421092    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:12:32.421288 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.421104    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:12:32.424405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.424383    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:12:32.424531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.424403    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4vfcf\""
Sep 10 14:12:32.424605 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.424559    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:12:32.425050 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.425026    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:12:32.426117 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.425482    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jzwbc\""
Sep 10 14:12:32.426117 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.425855    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.427264    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/da042357-c42d-4a96-970c-1ad9e2bbee80-config-volume\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.427306    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da042357-c42d-4a96-970c-1ad9e2bbee80-tmp-dir\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.427336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456e19b-d821-4774-9c39-ce6bc666bc6d-cert\") pod \"ingress-canary-xs8p9\" (UID: \"f456e19b-d821-4774-9c39-ce6bc666bc6d\") " pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.427390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlvt5\" (UniqueName: \"kubernetes.io/projected/da042357-c42d-4a96-970c-1ad9e2bbee80-kube-api-access-dlvt5\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.427454    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.427485    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xrqn\" (UniqueName: \"kubernetes.io/projected/f456e19b-d821-4774-9c39-ce6bc666bc6d-kube-api-access-6xrqn\") pod \"ingress-canary-xs8p9\" (UID: \"f456e19b-d821-4774-9c39-ce6bc666bc6d\") " pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.428026    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.428101    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:32.928080547 +0000 UTC m=+35.016538116 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.428529    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/da042357-c42d-4a96-970c-1ad9e2bbee80-config-volume\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.428698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.428567    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da042357-c42d-4a96-970c-1ad9e2bbee80-tmp-dir\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.432794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.432626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:32.433028 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.433010    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456e19b-d821-4774-9c39-ce6bc666bc6d-cert\") pod \"ingress-canary-xs8p9\" (UID: \"f456e19b-d821-4774-9c39-ce6bc666bc6d\") " pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.443317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.441590    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlvt5\" (UniqueName: \"kubernetes.io/projected/da042357-c42d-4a96-970c-1ad9e2bbee80-kube-api-access-dlvt5\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.443317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.441956    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xrqn\" (UniqueName: \"kubernetes.io/projected/f456e19b-d821-4774-9c39-ce6bc666bc6d-kube-api-access-6xrqn\") pod \"ingress-canary-xs8p9\" (UID: \"f456e19b-d821-4774-9c39-ce6bc666bc6d\") " pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.533671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.533637    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:32.589758 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.589719    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-xs8p9"
Sep 10 14:12:32.880328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.880247    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-xs8p9"]
Sep 10 14:12:32.884366 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:12:32.884339    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf456e19b_d821_4774_9c39_ce6bc666bc6d.slice/crio-ff0606562331f1dad025bfad2ce63031c4592f3f2873ffec033bb0d6117d0f82 WatchSource:0}: Error finding container ff0606562331f1dad025bfad2ce63031c4592f3f2873ffec033bb0d6117d0f82: Status 404 returned error can't find the container with id ff0606562331f1dad025bfad2ce63031c4592f3f2873ffec033bb0d6117d0f82
Sep 10 14:12:32.932346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:32.932313    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:32.932540 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.932473    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:32.932599 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:32.932547    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:33.932527864 +0000 UTC m=+36.020985418 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:33.603713 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:33.603673    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xs8p9" event={"ID":"f456e19b-d821-4774-9c39-ce6bc666bc6d","Type":"ContainerStarted","Data":"ff0606562331f1dad025bfad2ce63031c4592f3f2873ffec033bb0d6117d0f82"}
Sep 10 14:12:33.939951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:33.939909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:33.940135 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:33.940041    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:33.940135 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:33.940119    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:35.94009974 +0000 UTC m=+38.028557289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:35.955548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:35.955462    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:35.956059 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:35.955630    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:35.956059 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:35.955701    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:39.955684403 +0000 UTC m=+42.044141958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:36.614331 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:36.614042    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6fcde7-26f5-4234-a09a-f762fcdac8e4" containerID="ded3046d87c6e954cc2d1105d6a3bf3ff72eaf9e7f21d1228e33fa6ff41cbfee" exitCode=0
Sep 10 14:12:36.614331 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:36.614122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerDied","Data":"ded3046d87c6e954cc2d1105d6a3bf3ff72eaf9e7f21d1228e33fa6ff41cbfee"}
Sep 10 14:12:36.615980 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:36.615953    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-xs8p9" event={"ID":"f456e19b-d821-4774-9c39-ce6bc666bc6d","Type":"ContainerStarted","Data":"3a59165877bd6363ffcaba737cf6d9b1aba12c3a47ec4f1f03eaed16b77de432"}
Sep 10 14:12:36.656587 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:36.656513    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-xs8p9" podStartSLOduration=1.228541832 podStartE2EDuration="4.656495048s" totalImagesPullingTime="3.427953216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:36.652829835 +0000 UTC m=+38.741287383" watchObservedRunningTime="2026-09-10 14:12:36.656495048 +0000 UTC m=+38.744952619"
Sep 10 14:12:36.849997 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:36.849964    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:36.921105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:36.921079    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:37.621626 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:37.621592    2576 generic.go:356] "Generic (PLEG): container finished" podID="9b6fcde7-26f5-4234-a09a-f762fcdac8e4" containerID="f937e54b70442754905fca4bbe62928d2e3288c7750e39d4590bdfc38ed4ea07" exitCode=0
Sep 10 14:12:37.622157 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:37.621690    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerDied","Data":"f937e54b70442754905fca4bbe62928d2e3288c7750e39d4590bdfc38ed4ea07"}
Sep 10 14:12:38.627700 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:38.627666    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" event={"ID":"9b6fcde7-26f5-4234-a09a-f762fcdac8e4","Type":"ContainerStarted","Data":"3ca68703aca3867018a0d41d8fcc2aa9529ab4837ef47833e302d76ee4f1adc7"}
Sep 10 14:12:38.652679 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:38.652642    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hwkkb" podStartSLOduration=8.305589197 podStartE2EDuration="40.652627549s" totalImagesPullingTime="32.347038352s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:12:38.651190622 +0000 UTC m=+40.739648193" watchObservedRunningTime="2026-09-10 14:12:38.652627549 +0000 UTC m=+40.741085113"
Sep 10 14:12:38.773298 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:38.773264    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:38.836894 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:38.836858    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:39.982727 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:39.982686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:39.983150 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:39.982840    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:39.983150 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:39.982917    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:12:47.982902653 +0000 UTC m=+50.071360205 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:41.682545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:41.682512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:41.739300 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:41.739252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:12:48.037788 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:48.037749    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:12:48.038202 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:48.037921    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:12:48.038202 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:12:48.037988    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:04.037969785 +0000 UTC m=+66.126427353 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:12:58.946485 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:58.946455    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-tnzpc"
Sep 10 14:12:59.118633 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:12:59.118604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:00.775491 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:00.775451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:04.050913 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.050872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:13:04.051333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.050941    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:13:04.051333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.050976    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:13:04.051333 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:04.051129    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:13:04.051333 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:04.051208    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:13:36.051187134 +0000 UTC m=+98.139644684 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:13:04.054030 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.054011    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:13:04.054568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.054554    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:13:04.061328 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:04.061309    2576 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 14:13:04.061416 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:04.061396    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs podName:745510db-ad2d-4b22-97b7-4224f10153d5 nodeName:}" failed. No retries permitted until 2026-09-10 14:14:08.061374305 +0000 UTC m=+130.149831868 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs") pod "network-metrics-daemon-sr27l" (UID: "745510db-ad2d-4b22-97b7-4224f10153d5") : secret "metrics-daemon-secret" not found
Sep 10 14:13:04.064142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.064125    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/652327f0-1aef-4f03-b450-d89fcacee365-original-pull-secret\") pod \"global-pull-secret-syncer-qwlw4\" (UID: \"652327f0-1aef-4f03-b450-d89fcacee365\") " pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:13:04.151772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.151740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:13:04.154576 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.154560    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:13:04.165374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.165353    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:13:04.176020 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.176000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrnc9\" (UniqueName: \"kubernetes.io/projected/59a86a34-c66c-4e51-95d6-4632f72f0454-kube-api-access-xrnc9\") pod \"network-check-target-64znd\" (UID: \"59a86a34-c66c-4e51-95d6-4632f72f0454\") " pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:13:04.249192 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.249149    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qwlw4"
Sep 10 14:13:04.264541 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.264519    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4vfcf\""
Sep 10 14:13:04.271857 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.271834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:13:04.525165 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.525130    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qwlw4"]
Sep 10 14:13:04.529469 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:13:04.529442    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod652327f0_1aef_4f03_b450_d89fcacee365.slice/crio-873b6acd7eae75bfe2352a49f1e018d7d0fe21a6bbb33e2b9edd62984424d267 WatchSource:0}: Error finding container 873b6acd7eae75bfe2352a49f1e018d7d0fe21a6bbb33e2b9edd62984424d267: Status 404 returned error can't find the container with id 873b6acd7eae75bfe2352a49f1e018d7d0fe21a6bbb33e2b9edd62984424d267
Sep 10 14:13:04.554433 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.554404    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-64znd"]
Sep 10 14:13:04.557600 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:13:04.557575    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59a86a34_c66c_4e51_95d6_4632f72f0454.slice/crio-6a56c297dfd5c5a21746b9be5a6ca5446d8f170a7da728d0498fe5ec85d6c7e3 WatchSource:0}: Error finding container 6a56c297dfd5c5a21746b9be5a6ca5446d8f170a7da728d0498fe5ec85d6c7e3: Status 404 returned error can't find the container with id 6a56c297dfd5c5a21746b9be5a6ca5446d8f170a7da728d0498fe5ec85d6c7e3
Sep 10 14:13:04.690496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.690456    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-64znd" event={"ID":"59a86a34-c66c-4e51-95d6-4632f72f0454","Type":"ContainerStarted","Data":"6a56c297dfd5c5a21746b9be5a6ca5446d8f170a7da728d0498fe5ec85d6c7e3"}
Sep 10 14:13:04.694279 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:04.694251    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qwlw4" event={"ID":"652327f0-1aef-4f03-b450-d89fcacee365","Type":"ContainerStarted","Data":"873b6acd7eae75bfe2352a49f1e018d7d0fe21a6bbb33e2b9edd62984424d267"}
Sep 10 14:13:10.712079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:10.712034    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qwlw4" event={"ID":"652327f0-1aef-4f03-b450-d89fcacee365","Type":"ContainerStarted","Data":"5d82a1ef98327f9fd23086b6629ea1bcaedde9ad7b70da1b034355aa2031708f"}
Sep 10 14:13:10.713421 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:10.713398    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-64znd" event={"ID":"59a86a34-c66c-4e51-95d6-4632f72f0454","Type":"ContainerStarted","Data":"af3fec1d13454eb63c2dbd68fd1f522a865843796e263672587ba1de720f3138"}
Sep 10 14:13:10.713553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:10.713533    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:13:10.766489 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:10.766434    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-64znd" podStartSLOduration=67.472610441 podStartE2EDuration="1m12.766417478s" totalImagesPullingTime="5.293807037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:10.765594232 +0000 UTC m=+72.854051815" watchObservedRunningTime="2026-09-10 14:13:10.766417478 +0000 UTC m=+72.854875048"
Sep 10 14:13:10.766962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:10.766930    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qwlw4" podStartSLOduration=67.453584861 podStartE2EDuration="1m12.766918856s" totalImagesPullingTime="5.313333995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:10.741521947 +0000 UTC m=+72.829979519" watchObservedRunningTime="2026-09-10 14:13:10.766918856 +0000 UTC m=+72.855376436"
Sep 10 14:13:28.459004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.458885    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-qhh2g"]
Sep 10 14:13:28.461503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.461488    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.466663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.466636    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 10 14:13:28.467010 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.466991    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:13:28.467111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.467053    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 10 14:13:28.467775 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.467314    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 10 14:13:28.467775 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.467329    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-j4r7h\""
Sep 10 14:13:28.470833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.470813    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 10 14:13:28.484115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.484085    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-qhh2g"]
Sep 10 14:13:28.511360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.511319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkwn4\" (UniqueName: \"kubernetes.io/projected/af20a173-9bc9-4e04-9074-8aa13acadbd5-kube-api-access-zkwn4\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.511546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.511394    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af20a173-9bc9-4e04-9074-8aa13acadbd5-serving-cert\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.511546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.511420    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af20a173-9bc9-4e04-9074-8aa13acadbd5-trusted-ca\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.511546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.511446    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af20a173-9bc9-4e04-9074-8aa13acadbd5-config\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.611802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.611762    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af20a173-9bc9-4e04-9074-8aa13acadbd5-serving-cert\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.611802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.611799    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af20a173-9bc9-4e04-9074-8aa13acadbd5-trusted-ca\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.612000 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.611934    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af20a173-9bc9-4e04-9074-8aa13acadbd5-config\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.612067 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.612048    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkwn4\" (UniqueName: \"kubernetes.io/projected/af20a173-9bc9-4e04-9074-8aa13acadbd5-kube-api-access-zkwn4\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.612667 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.612627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/af20a173-9bc9-4e04-9074-8aa13acadbd5-config\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.612773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.612663    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/af20a173-9bc9-4e04-9074-8aa13acadbd5-trusted-ca\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.614257 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.614216    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/af20a173-9bc9-4e04-9074-8aa13acadbd5-serving-cert\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.644895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.644872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkwn4\" (UniqueName: \"kubernetes.io/projected/af20a173-9bc9-4e04-9074-8aa13acadbd5-kube-api-access-zkwn4\") pod \"console-operator-85b454874b-qhh2g\" (UID: \"af20a173-9bc9-4e04-9074-8aa13acadbd5\") " pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.770285 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.770158    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:28.808726 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.808691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:28.885954 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:28.885910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:29.044682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:29.044601    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-qhh2g"]
Sep 10 14:13:29.050010 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:13:29.049985    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf20a173_9bc9_4e04_9074_8aa13acadbd5.slice/crio-374d41f308f8db963a21f271bc10338c883597af379d69430504544ab48f7139 WatchSource:0}: Error finding container 374d41f308f8db963a21f271bc10338c883597af379d69430504544ab48f7139: Status 404 returned error can't find the container with id 374d41f308f8db963a21f271bc10338c883597af379d69430504544ab48f7139
Sep 10 14:13:29.760586 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:29.760549    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" event={"ID":"af20a173-9bc9-4e04-9074-8aa13acadbd5","Type":"ContainerStarted","Data":"374d41f308f8db963a21f271bc10338c883597af379d69430504544ab48f7139"}
Sep 10 14:13:31.767422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:31.767392    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/0.log"
Sep 10 14:13:31.767792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:31.767439    2576 generic.go:356] "Generic (PLEG): container finished" podID="af20a173-9bc9-4e04-9074-8aa13acadbd5" containerID="38466b4d7d38cf3933d848f67e0ce2dfa889527c9b492739b0ef08fab1d5f44b" exitCode=255
Sep 10 14:13:31.767792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:31.767469    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" event={"ID":"af20a173-9bc9-4e04-9074-8aa13acadbd5","Type":"ContainerDied","Data":"38466b4d7d38cf3933d848f67e0ce2dfa889527c9b492739b0ef08fab1d5f44b"}
Sep 10 14:13:31.767792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:31.767711    2576 scope.go:117] "RemoveContainer" containerID="38466b4d7d38cf3933d848f67e0ce2dfa889527c9b492739b0ef08fab1d5f44b"
Sep 10 14:13:32.771779 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.771751    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:13:32.772198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.772180    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/0.log"
Sep 10 14:13:32.772259 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.772214    2576 generic.go:356] "Generic (PLEG): container finished" podID="af20a173-9bc9-4e04-9074-8aa13acadbd5" containerID="27c76e6e4c145d269866af14d9b7d9fdfd792541540e77d19e13c3bd608847fb" exitCode=255
Sep 10 14:13:32.772304 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.772274    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" event={"ID":"af20a173-9bc9-4e04-9074-8aa13acadbd5","Type":"ContainerDied","Data":"27c76e6e4c145d269866af14d9b7d9fdfd792541540e77d19e13c3bd608847fb"}
Sep 10 14:13:32.772342 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.772328    2576 scope.go:117] "RemoveContainer" containerID="38466b4d7d38cf3933d848f67e0ce2dfa889527c9b492739b0ef08fab1d5f44b"
Sep 10 14:13:32.772571 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.772551    2576 scope.go:117] "RemoveContainer" containerID="27c76e6e4c145d269866af14d9b7d9fdfd792541540e77d19e13c3bd608847fb"
Sep 10 14:13:32.772753 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:32.772735    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-qhh2g_openshift-console-operator(af20a173-9bc9-4e04-9074-8aa13acadbd5)\"" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" podUID="af20a173-9bc9-4e04-9074-8aa13acadbd5"
Sep 10 14:13:32.903798 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.903766    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"]
Sep 10 14:13:32.904430 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.904409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:32.906177 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.906163    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"
Sep 10 14:13:32.908616 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.908597    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 10 14:13:32.908706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.908598    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 10 14:13:32.908890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.908877    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kbdn7\""
Sep 10 14:13:32.916829 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.916807    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"]
Sep 10 14:13:32.942811 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.942785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62hm7\" (UniqueName: \"kubernetes.io/projected/bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64-kube-api-access-62hm7\") pod \"migrator-fdbb8fdff-8bwt8\" (UID: \"bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"
Sep 10 14:13:32.994024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:32.993996    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:33.043210 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.043139    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62hm7\" (UniqueName: \"kubernetes.io/projected/bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64-kube-api-access-62hm7\") pod \"migrator-fdbb8fdff-8bwt8\" (UID: \"bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"
Sep 10 14:13:33.060465 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.060437    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62hm7\" (UniqueName: \"kubernetes.io/projected/bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64-kube-api-access-62hm7\") pod \"migrator-fdbb8fdff-8bwt8\" (UID: \"bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"
Sep 10 14:13:33.097471 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.097431    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:33.160460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.160432    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-gzp7p"]
Sep 10 14:13:33.162435 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.162421    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.164764 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.164738    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 14:13:33.164870 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.164738    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 14:13:33.165083 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.165067    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 14:13:33.165160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.165106    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 14:13:33.165160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.165136    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2mhjl\""
Sep 10 14:13:33.173674 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.173655    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gzp7p"]
Sep 10 14:13:33.215623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.215592    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"
Sep 10 14:13:33.236056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.236014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:33.244056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.244035    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-crio-socket\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.244156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.244065    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.244156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.244114    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.244156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.244149    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz9xk\" (UniqueName: \"kubernetes.io/projected/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-api-access-tz9xk\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.244284 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.244184    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-data-volume\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345424    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345471    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz9xk\" (UniqueName: \"kubernetes.io/projected/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-api-access-tz9xk\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345685 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.345542    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.345685 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345600    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-data-volume\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345685 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.345617    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:33.845601063 +0000 UTC m=+95.934058639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.345685 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345643    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-crio-socket\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345685 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345841 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345753    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-crio-socket\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.345841 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.345775    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.345841 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.345815    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:33.845806704 +0000 UTC m=+95.934264252 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.345937 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.345854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-data-volume\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.353778 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.353751    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz9xk\" (UniqueName: \"kubernetes.io/projected/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-api-access-tz9xk\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.481962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.481925    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8"]
Sep 10 14:13:33.485676 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.485636    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:33.486308 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:13:33.486285    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc8f4b28_79a9_4f5d_bb9f_9c95b71fdb64.slice/crio-46948aa66fec515800fef95a86de77950cab3d15f36a3dd38fc5e2c95c6fee4c WatchSource:0}: Error finding container 46948aa66fec515800fef95a86de77950cab3d15f36a3dd38fc5e2c95c6fee4c: Status 404 returned error can't find the container with id 46948aa66fec515800fef95a86de77950cab3d15f36a3dd38fc5e2c95c6fee4c
Sep 10 14:13:33.633303 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.633207    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:33.775676 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.775643    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8" event={"ID":"bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64","Type":"ContainerStarted","Data":"46948aa66fec515800fef95a86de77950cab3d15f36a3dd38fc5e2c95c6fee4c"}
Sep 10 14:13:33.777023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.777001    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:13:33.777316 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.777299    2576 scope.go:117] "RemoveContainer" containerID="27c76e6e4c145d269866af14d9b7d9fdfd792541540e77d19e13c3bd608847fb"
Sep 10 14:13:33.777447 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.777431    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-qhh2g_openshift-console-operator(af20a173-9bc9-4e04-9074-8aa13acadbd5)\"" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" podUID="af20a173-9bc9-4e04-9074-8aa13acadbd5"
Sep 10 14:13:33.850247 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.850206    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.850436 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.850330    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:33.850436 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.850336    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.850436 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.850397    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:34.850380321 +0000 UTC m=+96.938837892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:33.850627 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.850452    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.850627 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:33.850531    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:34.850511756 +0000 UTC m=+96.938969340 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:33.902020 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:33.901937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:34.226984 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.226933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:34.322744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.322702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:34.782922 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.782895    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8" event={"ID":"bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64","Type":"ContainerStarted","Data":"2542ffe35630bd72fd58e5d67712f8d4c5b89a05bb8f57b64aa346214b5975a6"}
Sep 10 14:13:34.783267 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.782949    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8" event={"ID":"bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64","Type":"ContainerStarted","Data":"e15a91f4559f4d6467879c24253a25ea375991745b65645e200639ce71f8a451"}
Sep 10 14:13:34.799811 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.799767    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-8bwt8" podStartSLOduration=1.635578952 podStartE2EDuration="2.799751372s" totalImagesPullingTime="1.16417242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:34.798578611 +0000 UTC m=+96.887036193" watchObservedRunningTime="2026-09-10 14:13:34.799751372 +0000 UTC m=+96.888208969"
Sep 10 14:13:34.856809 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.856777    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:34.856919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.856841    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:34.856919 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:34.856915    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:34.857008 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:34.856931    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:34.857008 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:34.856963    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:36.856951162 +0000 UTC m=+98.945408710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:34.857008 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:34.856995    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:36.856976704 +0000 UTC m=+98.945434253 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:34.886527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:34.886500    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:35.031402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.031374    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"]
Sep 10 14:13:35.033121 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.033106    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"
Sep 10 14:13:35.036936 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.036903    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:13:35.036936 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.036937    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 10 14:13:35.037120 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.037029    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-ss6d5\""
Sep 10 14:13:35.044492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.044472    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"]
Sep 10 14:13:35.062329 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.062299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqhbg\" (UniqueName: \"kubernetes.io/projected/5238c56a-d8ba-4307-95d5-e17c1eb00933-kube-api-access-vqhbg\") pod \"volume-data-source-validator-695c895bd6-bnmsv\" (UID: \"5238c56a-d8ba-4307-95d5-e17c1eb00933\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"
Sep 10 14:13:35.110007 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.109977    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:35.163621 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.163588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqhbg\" (UniqueName: \"kubernetes.io/projected/5238c56a-d8ba-4307-95d5-e17c1eb00933-kube-api-access-vqhbg\") pod \"volume-data-source-validator-695c895bd6-bnmsv\" (UID: \"5238c56a-d8ba-4307-95d5-e17c1eb00933\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"
Sep 10 14:13:35.173097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.173068    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqhbg\" (UniqueName: \"kubernetes.io/projected/5238c56a-d8ba-4307-95d5-e17c1eb00933-kube-api-access-vqhbg\") pod \"volume-data-source-validator-695c895bd6-bnmsv\" (UID: \"5238c56a-d8ba-4307-95d5-e17c1eb00933\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"
Sep 10 14:13:35.341808 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.341706    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"
Sep 10 14:13:35.609482 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.609406    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv"]
Sep 10 14:13:35.613619 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:13:35.613588    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5238c56a_d8ba_4307_95d5_e17c1eb00933.slice/crio-c33b69088787508d1870823d02d87f5c934ec398afafa46b6aaa3b0b064e3305 WatchSource:0}: Error finding container c33b69088787508d1870823d02d87f5c934ec398afafa46b6aaa3b0b064e3305: Status 404 returned error can't find the container with id c33b69088787508d1870823d02d87f5c934ec398afafa46b6aaa3b0b064e3305
Sep 10 14:13:35.777714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.777676    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-nnbff"]
Sep 10 14:13:35.780460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.780439    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.783545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.783519    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 10 14:13:35.784572 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.784535    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 10 14:13:35.784837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.784819    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7xh7c\""
Sep 10 14:13:35.784837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.784835    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 10 14:13:35.784958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.784821    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 10 14:13:35.787943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.787918    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv" event={"ID":"5238c56a-d8ba-4307-95d5-e17c1eb00933","Type":"ContainerStarted","Data":"c33b69088787508d1870823d02d87f5c934ec398afafa46b6aaa3b0b064e3305"}
Sep 10 14:13:35.789665 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.789643    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-nnbff"]
Sep 10 14:13:35.867333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.867215    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73b41d38-5161-4cf6-92b2-10884bb6bdf2-signing-cabundle\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.867333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.867288    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73b41d38-5161-4cf6-92b2-10884bb6bdf2-signing-key\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.867511 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.867397    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wskmd\" (UniqueName: \"kubernetes.io/projected/73b41d38-5161-4cf6-92b2-10884bb6bdf2-kube-api-access-wskmd\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.967902 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.967861    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73b41d38-5161-4cf6-92b2-10884bb6bdf2-signing-cabundle\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.967902 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.967899    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73b41d38-5161-4cf6-92b2-10884bb6bdf2-signing-key\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.968106 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.968043    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wskmd\" (UniqueName: \"kubernetes.io/projected/73b41d38-5161-4cf6-92b2-10884bb6bdf2-kube-api-access-wskmd\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.968693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.968676    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/73b41d38-5161-4cf6-92b2-10884bb6bdf2-signing-cabundle\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.970412 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.970396    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/73b41d38-5161-4cf6-92b2-10884bb6bdf2-signing-key\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.977370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.977346    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wskmd\" (UniqueName: \"kubernetes.io/projected/73b41d38-5161-4cf6-92b2-10884bb6bdf2-kube-api-access-wskmd\") pod \"service-ca-75d99c484-nnbff\" (UID: \"73b41d38-5161-4cf6-92b2-10884bb6bdf2\") " pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:35.994525 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:35.994497    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:36.068836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.068794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:13:36.068996 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:36.068952    2576 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:13:36.069035 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:36.069012    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls podName:da042357-c42d-4a96-970c-1ad9e2bbee80 nodeName:}" failed. No retries permitted until 2026-09-10 14:14:40.06899743 +0000 UTC m=+162.157454979 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls") pod "dns-default-bxrf8" (UID: "da042357-c42d-4a96-970c-1ad9e2bbee80") : secret "dns-default-metrics-tls" not found
Sep 10 14:13:36.090391 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.090353    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-nnbff"
Sep 10 14:13:36.120907 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.120832    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4lmh_7b327197-ec2d-4799-9164-bbea8925c918/dns-node-resolver/0.log"
Sep 10 14:13:36.235265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.235217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:36.361447 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.361418    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-nnbff"]
Sep 10 14:13:36.365596 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:13:36.365569    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b41d38_5161_4cf6_92b2_10884bb6bdf2.slice/crio-a7eeff4f6c08d6357d24f66fc474ba06d9c677e4199199f63420104dc2b68d74 WatchSource:0}: Error finding container a7eeff4f6c08d6357d24f66fc474ba06d9c677e4199199f63420104dc2b68d74: Status 404 returned error can't find the container with id a7eeff4f6c08d6357d24f66fc474ba06d9c677e4199199f63420104dc2b68d74
Sep 10 14:13:36.523782 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.523750    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4spz6_c22a5c55-fbd5-4a43-bf63-0ec6b132b0be/node-ca/0.log"
Sep 10 14:13:36.793159 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.793126    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-nnbff" event={"ID":"73b41d38-5161-4cf6-92b2-10884bb6bdf2","Type":"ContainerStarted","Data":"a7eeff4f6c08d6357d24f66fc474ba06d9c677e4199199f63420104dc2b68d74"}
Sep 10 14:13:36.874988 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.874952    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:36.875141 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:36.875003    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:36.875141 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:36.875088    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:36.875141 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:36.875098    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:36.875141 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:36.875138    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:40.875125578 +0000 UTC m=+102.963583127 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:36.875336 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:36.875160    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:40.875144663 +0000 UTC m=+102.963602212 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:37.723297 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:37.723266    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xs8p9_f456e19b-d821-4774-9c39-ce6bc666bc6d/serve-healthcheck-canary/0.log"
Sep 10 14:13:37.798615 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:37.798580    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv" event={"ID":"5238c56a-d8ba-4307-95d5-e17c1eb00933","Type":"ContainerStarted","Data":"73b96f1fd863718bdfe746397a747ef000f723aa9e820054cdef16cd98f99276"}
Sep 10 14:13:37.921474 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:37.921435    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8bwt8_bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64/migrator/0.log"
Sep 10 14:13:38.021101 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.021015    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:38.120341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.120311    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8bwt8_bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64/graceful-termination/0.log"
Sep 10 14:13:38.136706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.136670    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:38.581422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.581386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:38.770417 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.770388    2576 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:38.770566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.770445    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:38.770817 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.770803    2576 scope.go:117] "RemoveContainer" containerID="27c76e6e4c145d269866af14d9b7d9fdfd792541540e77d19e13c3bd608847fb"
Sep 10 14:13:38.771010 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:38.770990    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-qhh2g_openshift-console-operator(af20a173-9bc9-4e04-9074-8aa13acadbd5)\"" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" podUID="af20a173-9bc9-4e04-9074-8aa13acadbd5"
Sep 10 14:13:38.803448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.803412    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-nnbff" event={"ID":"73b41d38-5161-4cf6-92b2-10884bb6bdf2","Type":"ContainerStarted","Data":"e7a1ac7d7c79f2ee0135672099000c8f7ac72975f507770dc0f336a0125d31b7"}
Sep 10 14:13:38.825887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.825836    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-nnbff" podStartSLOduration=1.55867511 podStartE2EDuration="3.82582101s" totalImagesPullingTime="2.2671459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:38.825630129 +0000 UTC m=+100.914087711" watchObservedRunningTime="2026-09-10 14:13:38.82582101 +0000 UTC m=+100.914278582"
Sep 10 14:13:38.826437 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:38.826412    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-bnmsv" podStartSLOduration=2.66657493 podStartE2EDuration="3.826405518s" totalImagesPullingTime="1.159830588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:37.832494124 +0000 UTC m=+99.920951694" watchObservedRunningTime="2026-09-10 14:13:38.826405518 +0000 UTC m=+100.914863092"
Sep 10 14:13:40.153618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:40.153580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:40.233769 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:40.233735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:40.751431 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:40.751396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:40.904480 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:40.904439    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:40.904646 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:40.904498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:40.904646 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:40.904576    2576 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:13:40.904646 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:40.904582    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:40.904646 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:40.904638    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:48.904624483 +0000 UTC m=+110.993082033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : configmap "kube-rbac-proxy" not found
Sep 10 14:13:40.904893 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:40.904652    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:13:48.904646308 +0000 UTC m=+110.993103857 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:41.396755 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:41.396724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:41.718378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:41.718346    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-64znd"
Sep 10 14:13:41.789877 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:41.789845    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:41.961460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:41.961426    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:42.385515 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:42.385473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:42.768810 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:42.768777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:43.215939 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:43.215906    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:43.608764 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:43.608678    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:43.918291 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:43.918194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:44.045138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:44.045104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:44.428276 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:44.428242    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:44.585969 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:44.585941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:44.754197 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:44.754115    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:44.940975 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:44.940936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:45.085715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:45.085641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:45.844895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:45.844863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:46.182294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:46.182262    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:46.309498 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:46.309466    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:46.424888 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:46.424857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:46.610775 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:46.610687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:46.826086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:46.826049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:46.977955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:46.977924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:47.965974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:47.965936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:48.396386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:48.396304    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:48.562627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:48.562597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:48.794744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:48.794716    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:48.970980 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:48.970939    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:48.971413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:48.971007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:48.971413 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:48.971094    2576 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:48.971413 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:13:48.971182    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls podName:3842435a-3a63-4c3d-b98b-a6f15b4e53fe nodeName:}" failed. No retries permitted until 2026-09-10 14:14:04.971160068 +0000 UTC m=+127.059617617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls") pod "insights-runtime-extractor-gzp7p" (UID: "3842435a-3a63-4c3d-b98b-a6f15b4e53fe") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:13:48.971601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:48.971584    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:13:49.050210 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:49.050144    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:49.762437 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:49.762400    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:50.160667 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:50.160558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:50.365223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:50.365186    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:50.594468 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:50.594433    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:51.275464 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:51.275430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:52.416056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:52.416026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:52.514775 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:52.514742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:54.421549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:54.421519    2576 scope.go:117] "RemoveContainer" containerID="27c76e6e4c145d269866af14d9b7d9fdfd792541540e77d19e13c3bd608847fb"
Sep 10 14:13:54.851981 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:54.851904    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:13:54.851981 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:54.851969    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" event={"ID":"af20a173-9bc9-4e04-9074-8aa13acadbd5","Type":"ContainerStarted","Data":"cd9db6c30e3450c18148e9040d59afe6bde08925843d0dd3e2fb5bde032d1a7c"}
Sep 10 14:13:54.852313 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:54.852291    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:54.875904 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:54.875855    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-qhh2g" podStartSLOduration=24.553175402 podStartE2EDuration="26.875839849s" totalImagesPullingTime="2.322664447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:13:54.875504905 +0000 UTC m=+116.963962477" watchObservedRunningTime="2026-09-10 14:13:54.875839849 +0000 UTC m=+116.964297420"
Sep 10 14:13:55.364247 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:55.364202    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-qhh2g"
Sep 10 14:13:55.600835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:55.600809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:55.718925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:55.718896    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:56.103587 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:56.103504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:56.241770 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:56.241726    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:56.319034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:56.319009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:56.474434 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:56.474397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:56.888784 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:56.888697    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:56.966418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:56.966390    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:57.472323 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:57.472281    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:57.648830 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:57.648797    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:57.970879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:57.970847    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:13:59.788122 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:13:59.788082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:01.958531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.958496    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-k4f7g"]
Sep 10 14:14:01.960892 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.960874    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g"
Sep 10 14:14:01.965290 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.965270    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xx7zl\""
Sep 10 14:14:01.976559 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.976531    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"]
Sep 10 14:14:01.981950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.981922    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:01.984994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.984972    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-k4f7g"]
Sep 10 14:14:01.985492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.985475    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 14:14:01.985743 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.985725    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 14:14:01.986070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.986049    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 14:14:01.986359 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.986344    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 14:14:01.986707 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.986686    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 14:14:01.986887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.986871    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 14:14:01.986935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:01.986909    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 14:14:02.007941 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.007905    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"]
Sep 10 14:14:02.069938 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.069908    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.069938 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.069941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhl4z\" (UniqueName: \"kubernetes.io/projected/23d38adf-1f2f-4e0d-aad1-15b61e468c4e-kube-api-access-jhl4z\") pod \"network-check-source-685456655f-k4f7g\" (UID: \"23d38adf-1f2f-4e0d-aad1-15b61e468c4e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g"
Sep 10 14:14:02.070147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.069994    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.070147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.070023    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwp45\" (UniqueName: \"kubernetes.io/projected/992e80ad-9c21-4b66-af1e-cd32138b4251-kube-api-access-qwp45\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.070147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.070080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-ca\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.070264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.070145    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e80ad-9c21-4b66-af1e-cd32138b4251-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.070264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.070193    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-hub\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.167885 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.167849    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5ccc597495-spmwc"]
Sep 10 14:14:02.169969 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.169953    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.170581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-hub\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.170692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170624    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.170692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhl4z\" (UniqueName: \"kubernetes.io/projected/23d38adf-1f2f-4e0d-aad1-15b61e468c4e-kube-api-access-jhl4z\") pod \"network-check-source-685456655f-k4f7g\" (UID: \"23d38adf-1f2f-4e0d-aad1-15b61e468c4e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g"
Sep 10 14:14:02.170692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.170914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170712    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwp45\" (UniqueName: \"kubernetes.io/projected/992e80ad-9c21-4b66-af1e-cd32138b4251-kube-api-access-qwp45\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.170914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170746    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-ca\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.170914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.170789    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e80ad-9c21-4b66-af1e-cd32138b4251-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.171569 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.171543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/992e80ad-9c21-4b66-af1e-cd32138b4251-additional-service-ca\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.173211 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.173183    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-ca\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.173714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.173698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.173805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.173745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-hub\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.173805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.173755    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/992e80ad-9c21-4b66-af1e-cd32138b4251-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.182168 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.182144    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 14:14:02.182324 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.182199    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-wzphp\""
Sep 10 14:14:02.182324 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.182199    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 14:14:02.182469 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.182390    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 14:14:02.185355 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.185334    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 14:14:02.194838 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.194806    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:02.205757 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.205726    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ccc597495-spmwc"]
Sep 10 14:14:02.212339 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.212279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhl4z\" (UniqueName: \"kubernetes.io/projected/23d38adf-1f2f-4e0d-aad1-15b61e468c4e-kube-api-access-jhl4z\") pod \"network-check-source-685456655f-k4f7g\" (UID: \"23d38adf-1f2f-4e0d-aad1-15b61e468c4e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g"
Sep 10 14:14:02.231794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.231754    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwp45\" (UniqueName: \"kubernetes.io/projected/992e80ad-9c21-4b66-af1e-cd32138b4251-kube-api-access-qwp45\") pod \"cluster-proxy-proxy-agent-54b55f59c8-dd2jk\" (UID: \"992e80ad-9c21-4b66-af1e-cd32138b4251\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.269506 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.269465    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g"
Sep 10 14:14:02.271849 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.271827    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f7b1f52-033f-4de4-9d35-d094b670e283-trusted-ca\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.271931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.271859    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f7b1f52-033f-4de4-9d35-d094b670e283-installation-pull-secrets\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.271931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.271882    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f7b1f52-033f-4de4-9d35-d094b670e283-ca-trust-extracted\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.271997 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.271943    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f7b1f52-033f-4de4-9d35-d094b670e283-image-registry-private-configuration\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.271997 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.271961    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f7b1f52-033f-4de4-9d35-d094b670e283-registry-certificates\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.271997 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.271987    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwmsp\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-kube-api-access-jwmsp\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.272090 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.272006    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-registry-tls\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.272090 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.272020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-bound-sa-token\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.302398 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.302369    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"
Sep 10 14:14:02.373286 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373259    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f7b1f52-033f-4de4-9d35-d094b670e283-registry-certificates\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373304    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwmsp\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-kube-api-access-jwmsp\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373333    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-registry-tls\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373351    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-bound-sa-token\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373410    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f7b1f52-033f-4de4-9d35-d094b670e283-trusted-ca\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373438    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f7b1f52-033f-4de4-9d35-d094b670e283-installation-pull-secrets\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373475    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f7b1f52-033f-4de4-9d35-d094b670e283-ca-trust-extracted\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.373561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.373530    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f7b1f52-033f-4de4-9d35-d094b670e283-image-registry-private-configuration\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.374163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.374114    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7f7b1f52-033f-4de4-9d35-d094b670e283-ca-trust-extracted\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.374373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.374186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7f7b1f52-033f-4de4-9d35-d094b670e283-registry-certificates\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.374506 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.374485    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7f7b1f52-033f-4de4-9d35-d094b670e283-trusted-ca\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.376502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.376478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7f7b1f52-033f-4de4-9d35-d094b670e283-installation-pull-secrets\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.377028 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.377004    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-registry-tls\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.377323 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.377305    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7f7b1f52-033f-4de4-9d35-d094b670e283-image-registry-private-configuration\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.389654 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.389619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwmsp\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-kube-api-access-jwmsp\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.390815 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.390793    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7f7b1f52-033f-4de4-9d35-d094b670e283-bound-sa-token\") pod \"image-registry-5ccc597495-spmwc\" (UID: \"7f7b1f52-033f-4de4-9d35-d094b670e283\") " pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.391284 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.391248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:02.486914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.486828    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.605460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.605436    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-k4f7g"]
Sep 10 14:14:02.606811 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:02.606790    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23d38adf_1f2f_4e0d_aad1_15b61e468c4e.slice/crio-75832caff1c9fd4603a85ea5671ed8db5211abeec10b616f8e5200b7d812141b WatchSource:0}: Error finding container 75832caff1c9fd4603a85ea5671ed8db5211abeec10b616f8e5200b7d812141b: Status 404 returned error can't find the container with id 75832caff1c9fd4603a85ea5671ed8db5211abeec10b616f8e5200b7d812141b
Sep 10 14:14:02.631558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.631533    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk"]
Sep 10 14:14:02.648115 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:02.648089    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod992e80ad_9c21_4b66_af1e_cd32138b4251.slice/crio-ae2328f4080103f155821e9503204a5099d90a95b447dea9d35722ea813876b6 WatchSource:0}: Error finding container ae2328f4080103f155821e9503204a5099d90a95b447dea9d35722ea813876b6: Status 404 returned error can't find the container with id ae2328f4080103f155821e9503204a5099d90a95b447dea9d35722ea813876b6
Sep 10 14:14:02.782399 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.782306    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5ccc597495-spmwc"]
Sep 10 14:14:02.786506 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:02.786482    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f7b1f52_033f_4de4_9d35_d094b670e283.slice/crio-e0dc0b6a178b33aac4cc3bb49b35df2782fe4de8a192b21aa3244a1cd15c52da WatchSource:0}: Error finding container e0dc0b6a178b33aac4cc3bb49b35df2782fe4de8a192b21aa3244a1cd15c52da: Status 404 returned error can't find the container with id e0dc0b6a178b33aac4cc3bb49b35df2782fe4de8a192b21aa3244a1cd15c52da
Sep 10 14:14:02.878648 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.878617    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ccc597495-spmwc" event={"ID":"7f7b1f52-033f-4de4-9d35-d094b670e283","Type":"ContainerStarted","Data":"357b64d88bef75bd7f0f5ac352ff669621bfdec66480edee4dd3b9defdb2443a"}
Sep 10 14:14:02.878804 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.878668    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5ccc597495-spmwc" event={"ID":"7f7b1f52-033f-4de4-9d35-d094b670e283","Type":"ContainerStarted","Data":"e0dc0b6a178b33aac4cc3bb49b35df2782fe4de8a192b21aa3244a1cd15c52da"}
Sep 10 14:14:02.878804 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.878707    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:02.879967 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.879933    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk" event={"ID":"992e80ad-9c21-4b66-af1e-cd32138b4251","Type":"ContainerStarted","Data":"ae2328f4080103f155821e9503204a5099d90a95b447dea9d35722ea813876b6"}
Sep 10 14:14:02.881629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.881603    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g" event={"ID":"23d38adf-1f2f-4e0d-aad1-15b61e468c4e","Type":"ContainerStarted","Data":"42fbbcf05a955f0ede1c7418d6e8edbab625b36c1bb6bc8c22ca641a2a33c782"}
Sep 10 14:14:02.881733 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.881649    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g" event={"ID":"23d38adf-1f2f-4e0d-aad1-15b61e468c4e","Type":"ContainerStarted","Data":"75832caff1c9fd4603a85ea5671ed8db5211abeec10b616f8e5200b7d812141b"}
Sep 10 14:14:02.907907 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.907862    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5ccc597495-spmwc" podStartSLOduration=0.90784831 podStartE2EDuration="907.84831ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:02.907139318 +0000 UTC m=+124.995596890" watchObservedRunningTime="2026-09-10 14:14:02.90784831 +0000 UTC m=+124.996305875"
Sep 10 14:14:02.926634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:02.926580    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-k4f7g" podStartSLOduration=1.926559751 podStartE2EDuration="1.926559751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:02.925808467 +0000 UTC m=+125.014266037" watchObservedRunningTime="2026-09-10 14:14:02.926559751 +0000 UTC m=+125.015017326"
Sep 10 14:14:03.174353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:03.174315    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:03.276060 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:03.276000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:03.388674 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:03.388644    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:03.894411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:03.894373    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:05.003785 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.003748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:14:05.006795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.006766    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3842435a-3a63-4c3d-b98b-a6f15b4e53fe-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-gzp7p\" (UID: \"3842435a-3a63-4c3d-b98b-a6f15b4e53fe\") " pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:14:05.155906 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.155863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:05.232856 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.232823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:05.273474 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.273402    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2mhjl\""
Sep 10 14:14:05.281596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.281536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-gzp7p"
Sep 10 14:14:05.946225 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:05.946184    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-gzp7p"]
Sep 10 14:14:06.301418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:06.301321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:06.395934 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:06.395869    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:06.470106 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:06.470071    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:07.046043 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:07.046006    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3842435a_3a63_4c3d_b98b_a6f15b4e53fe.slice/crio-fdbe35c45e580e2d4e9f48aa75ab267c5501b464e615220c655fa8cc1981a8d3 WatchSource:0}: Error finding container fdbe35c45e580e2d4e9f48aa75ab267c5501b464e615220c655fa8cc1981a8d3: Status 404 returned error can't find the container with id fdbe35c45e580e2d4e9f48aa75ab267c5501b464e615220c655fa8cc1981a8d3
Sep 10 14:14:07.901423 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:07.901389    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk" event={"ID":"992e80ad-9c21-4b66-af1e-cd32138b4251","Type":"ContainerStarted","Data":"b6917782917f03251e69ccefb83eb71e2afeca646beaa38b4ce603ec09f58969"}
Sep 10 14:14:07.901973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:07.901440    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk" event={"ID":"992e80ad-9c21-4b66-af1e-cd32138b4251","Type":"ContainerStarted","Data":"9c6637ab665381b69fb09c34a490eb813cacb34c6a0bc67df4ac04636353780f"}
Sep 10 14:14:07.901973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:07.901464    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk" event={"ID":"992e80ad-9c21-4b66-af1e-cd32138b4251","Type":"ContainerStarted","Data":"23eae0c2cf118eb759a493efa0a7755676c196c6d259f1e43fe67f39e089f9b2"}
Sep 10 14:14:07.902990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:07.902967    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzp7p" event={"ID":"3842435a-3a63-4c3d-b98b-a6f15b4e53fe","Type":"ContainerStarted","Data":"fa5d93b2d658d7d61c69de7f546ce206502b48bfd25c64d2fbd39f61980b4a6e"}
Sep 10 14:14:07.903078 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:07.903013    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzp7p" event={"ID":"3842435a-3a63-4c3d-b98b-a6f15b4e53fe","Type":"ContainerStarted","Data":"fdbe35c45e580e2d4e9f48aa75ab267c5501b464e615220c655fa8cc1981a8d3"}
Sep 10 14:14:07.924852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:07.924800    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-54b55f59c8-dd2jk" podStartSLOduration=2.484730918 podStartE2EDuration="6.924787839s" totalImagesPullingTime="4.440056921s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:14:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:07.922972813 +0000 UTC m=+130.011430384" watchObservedRunningTime="2026-09-10 14:14:07.924787839 +0000 UTC m=+130.013245410"
Sep 10 14:14:08.131829 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.131786    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:14:08.134611 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.134582    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/745510db-ad2d-4b22-97b7-4224f10153d5-metrics-certs\") pod \"network-metrics-daemon-sr27l\" (UID: \"745510db-ad2d-4b22-97b7-4224f10153d5\") " pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:14:08.158409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.158359    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jzwbc\""
Sep 10 14:14:08.165699 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.165684    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sr27l"
Sep 10 14:14:08.454017 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.453981    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sr27l"]
Sep 10 14:14:08.457952 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:08.457925    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745510db_ad2d_4b22_97b7_4224f10153d5.slice/crio-9ce912badb6dc29466b9c69dd94c2133e80eb003ddf1532d378072217ef45c8f WatchSource:0}: Error finding container 9ce912badb6dc29466b9c69dd94c2133e80eb003ddf1532d378072217ef45c8f: Status 404 returned error can't find the container with id 9ce912badb6dc29466b9c69dd94c2133e80eb003ddf1532d378072217ef45c8f
Sep 10 14:14:08.907800 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.907715    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sr27l" event={"ID":"745510db-ad2d-4b22-97b7-4224f10153d5","Type":"ContainerStarted","Data":"9ce912badb6dc29466b9c69dd94c2133e80eb003ddf1532d378072217ef45c8f"}
Sep 10 14:14:08.909700 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:08.909653    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzp7p" event={"ID":"3842435a-3a63-4c3d-b98b-a6f15b4e53fe","Type":"ContainerStarted","Data":"1c2fba7c7defa9c25f634a812b936fa54b18ddfa098b1668514fe900bc420020"}
Sep 10 14:14:09.278363 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:09.278330    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:09.346407 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:09.346354    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:09.886214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:09.886177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:10.919175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:10.919143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sr27l" event={"ID":"745510db-ad2d-4b22-97b7-4224f10153d5","Type":"ContainerStarted","Data":"1d506aeca69d08b6b619ec9fc2faebb3d3a561c68c1bcff2ec2d61b000076fd9"}
Sep 10 14:14:10.919656 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:10.919212    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sr27l" event={"ID":"745510db-ad2d-4b22-97b7-4224f10153d5","Type":"ContainerStarted","Data":"4c45cd165a95be1f3a793db78cee55de2e3d9d4076060bb369265476e2e25e32"}
Sep 10 14:14:10.921325 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:10.921304    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-gzp7p" event={"ID":"3842435a-3a63-4c3d-b98b-a6f15b4e53fe","Type":"ContainerStarted","Data":"6b0be200b46933899f1e3a4ca12fe0cb444f80c80e4a8abdd8c08f7165de1fb7"}
Sep 10 14:14:10.943898 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:10.943850    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sr27l" podStartSLOduration=131.437374754 podStartE2EDuration="2m12.943835987s" totalImagesPullingTime="1.506461233s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:10.942979082 +0000 UTC m=+133.031436653" watchObservedRunningTime="2026-09-10 14:14:10.943835987 +0000 UTC m=+133.032293559"
Sep 10 14:14:10.967649 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:10.967591    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-gzp7p" podStartSLOduration=35.186597465 podStartE2EDuration="37.967574958s" totalImagesPullingTime="2.780977493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:13:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:10.966711206 +0000 UTC m=+133.055168778" watchObservedRunningTime="2026-09-10 14:14:10.967574958 +0000 UTC m=+133.056032528"
Sep 10 14:14:11.087505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.087469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:11.190089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.190033    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:11.290208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.290171    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:11.398444 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.398413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:11.493616 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.493519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:11.695856 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.695820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:11.885049 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.884961    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9g74h"]
Sep 10 14:14:11.887507 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.887482    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:11.891446 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.891423    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 14:14:11.891596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.891566    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 14:14:11.895973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.895954    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 14:14:11.896271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.896223    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z2qt2\""
Sep 10 14:14:11.896390 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.896358    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 14:14:11.896484 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.896467    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 14:14:11.896659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.896643    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 14:14:11.924536 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:11.924504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.024006 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.023972    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.067380 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067344    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-textfile\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067462    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-tls\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-wtmp\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067567    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-sys\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067604    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdds\" (UniqueName: \"kubernetes.io/projected/fac3e327-1d2f-4765-8246-8f0c4c7a0917-kube-api-access-2vdds\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067638    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac3e327-1d2f-4765-8246-8f0c4c7a0917-metrics-client-ca\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067799    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-root\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.067957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.067857    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-accelerators-collector-config\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.126619 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.126586    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.168839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168737    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-root\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.168839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168800    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-accelerators-collector-config\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.168839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-textfile\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168856    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-root\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168872    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-tls\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168915    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-wtmp\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168955    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-sys\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168971    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdds\" (UniqueName: \"kubernetes.io/projected/fac3e327-1d2f-4765-8246-8f0c4c7a0917-kube-api-access-2vdds\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.168994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.169042    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-sys\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.169102    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac3e327-1d2f-4765-8246-8f0c4c7a0917-metrics-client-ca\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.169113    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-wtmp\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169607 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.169576    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-accelerators-collector-config\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.169700    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fac3e327-1d2f-4765-8246-8f0c4c7a0917-metrics-client-ca\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.169813 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.169791    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-textfile\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.171940 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.171917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.172308 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.172289    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fac3e327-1d2f-4765-8246-8f0c4c7a0917-node-exporter-tls\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.178357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.178334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdds\" (UniqueName: \"kubernetes.io/projected/fac3e327-1d2f-4765-8246-8f0c4c7a0917-kube-api-access-2vdds\") pod \"node-exporter-9g74h\" (UID: \"fac3e327-1d2f-4765-8246-8f0c4c7a0917\") " pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.197978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.197951    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9g74h"
Sep 10 14:14:12.208538 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:12.208508    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac3e327_1d2f_4765_8246_8f0c4c7a0917.slice/crio-f1bb2c7849239d2443b232cb7220663bf12721e163b4e88b304b484dc7a47644 WatchSource:0}: Error finding container f1bb2c7849239d2443b232cb7220663bf12721e163b4e88b304b484dc7a47644: Status 404 returned error can't find the container with id f1bb2c7849239d2443b232cb7220663bf12721e163b4e88b304b484dc7a47644
Sep 10 14:14:12.273279 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.273222    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.378844 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.378812    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.837649 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.837618    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.917700 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.917663    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:12.930384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:12.930354    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9g74h" event={"ID":"fac3e327-1d2f-4765-8246-8f0c4c7a0917","Type":"ContainerStarted","Data":"f1bb2c7849239d2443b232cb7220663bf12721e163b4e88b304b484dc7a47644"}
Sep 10 14:14:13.934847 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:13.934812    2576 generic.go:356] "Generic (PLEG): container finished" podID="fac3e327-1d2f-4765-8246-8f0c4c7a0917" containerID="abb56c9579ce4ab7ac16375b5ec4f1f2d819b636d0af5fb446a686b64153c9fe" exitCode=0
Sep 10 14:14:13.935213 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:13.934857    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9g74h" event={"ID":"fac3e327-1d2f-4765-8246-8f0c4c7a0917","Type":"ContainerDied","Data":"abb56c9579ce4ab7ac16375b5ec4f1f2d819b636d0af5fb446a686b64153c9fe"}
Sep 10 14:14:14.676070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:14.676040    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:14.747794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:14.747754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:14.940905 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:14.940814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9g74h" event={"ID":"fac3e327-1d2f-4765-8246-8f0c4c7a0917","Type":"ContainerStarted","Data":"9619646f297056909f3d768d6e12227fb9e69b1e410ab1d0b8261926536b88c2"}
Sep 10 14:14:14.940905 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:14.940889    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9g74h" event={"ID":"fac3e327-1d2f-4765-8246-8f0c4c7a0917","Type":"ContainerStarted","Data":"7eacb65a15d095321f2c8e59f66540bf86a5e935f93d41e2d21cdae66e436bab"}
Sep 10 14:14:14.976944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:14.976888    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:15.078254 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:15.078194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:16.474835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:16.474800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:16.549532 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:16.549492    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:18.226137 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:18.226106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:20.167298 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:20.167259    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:20.234395 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:20.234361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:22.491456 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:22.491425    2576 patch_prober.go:28] interesting pod/image-registry-5ccc597495-spmwc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 14:14:22.491833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:22.491484    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5ccc597495-spmwc" podUID="7f7b1f52-033f-4de4-9d35-d094b670e283" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:14:23.888960 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:23.888929    2576 patch_prober.go:28] interesting pod/image-registry-5ccc597495-spmwc container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 14:14:23.889374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:23.888978    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5ccc597495-spmwc" podUID="7f7b1f52-033f-4de4-9d35-d094b670e283" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:14:27.175669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:27.175637    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:27.235952 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:27.235922    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:27.300429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:27.300396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:29.635614 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:29.635576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:29.694448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:29.694410    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:29.975942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:29.975898    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:30.084343 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:30.084292    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:31.164404 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:31.161296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:31.225980 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:31.225937    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:31.883459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:31.883413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:31.984126 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:31.984097    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:32.240403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:32.240359    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:32.340828 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:32.340777    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:32.491260 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:32.491153    2576 patch_prober.go:28] interesting pod/image-registry-5ccc597495-spmwc container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 10 14:14:32.491260 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:32.491224    2576 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5ccc597495-spmwc" podUID="7f7b1f52-033f-4de4-9d35-d094b670e283" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:14:33.218735 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:33.218706    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:33.320580 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:33.320535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:33.888304 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:33.888277    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5ccc597495-spmwc"
Sep 10 14:14:33.911003 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:33.910959    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9g74h" podStartSLOduration=21.540454552 podStartE2EDuration="22.910942574s" totalImagesPullingTime="1.300635559s" totalInitContainerRuntime="69.852463ms" isStatefulPod=true podCreationTimestamp="2026-09-10 14:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:14.963133601 +0000 UTC m=+137.051591173" watchObservedRunningTime="2026-09-10 14:14:33.910942574 +0000 UTC m=+155.999400145"
Sep 10 14:14:34.023280 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:34.023221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:34.118429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:34.118401    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:34.219886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:34.219857    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:35.281994 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:14:35.281952    2576 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-bxrf8" podUID="da042357-c42d-4a96-970c-1ad9e2bbee80"
Sep 10 14:14:36.010125 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:36.010045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:14:37.426014 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:37.425968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:37.490877 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:37.490821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:39.056692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:39.056659    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:39.157445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:39.157402    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:40.089221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:40.089188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:14:40.091604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:40.091572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da042357-c42d-4a96-970c-1ad9e2bbee80-metrics-tls\") pod \"dns-default-bxrf8\" (UID: \"da042357-c42d-4a96-970c-1ad9e2bbee80\") " pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:14:40.213194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:40.213163    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xwswr\""
Sep 10 14:14:40.221449 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:40.221425    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:14:40.532027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:40.531989    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bxrf8"]
Sep 10 14:14:40.536404 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:14:40.536367    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda042357_c42d_4a96_970c_1ad9e2bbee80.slice/crio-3fcc7409bc319ff54feecb30379c4323f92bbb9d78fb12f559960ebf7ff0ae00 WatchSource:0}: Error finding container 3fcc7409bc319ff54feecb30379c4323f92bbb9d78fb12f559960ebf7ff0ae00: Status 404 returned error can't find the container with id 3fcc7409bc319ff54feecb30379c4323f92bbb9d78fb12f559960ebf7ff0ae00
Sep 10 14:14:41.029738 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:41.029706    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bxrf8" event={"ID":"da042357-c42d-4a96-970c-1ad9e2bbee80","Type":"ContainerStarted","Data":"3fcc7409bc319ff54feecb30379c4323f92bbb9d78fb12f559960ebf7ff0ae00"}
Sep 10 14:14:43.039109 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:43.039078    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bxrf8" event={"ID":"da042357-c42d-4a96-970c-1ad9e2bbee80","Type":"ContainerStarted","Data":"d8fa225544f6943836c7029755bf6e7047632675fe69b8b56963785a74bf79c0"}
Sep 10 14:14:43.039519 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:43.039137    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bxrf8" event={"ID":"da042357-c42d-4a96-970c-1ad9e2bbee80","Type":"ContainerStarted","Data":"f43f57dc2537311f1f09da13ff3ac0d1b65c15c47f1dbe88d413e21964ca8596"}
Sep 10 14:14:43.039519 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:43.039178    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:14:43.056548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:43.056504    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bxrf8" podStartSLOduration=129.039998209 podStartE2EDuration="2m11.056492082s" totalImagesPullingTime="2.016493873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:12:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:14:43.055140115 +0000 UTC m=+165.143597687" watchObservedRunningTime="2026-09-10 14:14:43.056492082 +0000 UTC m=+165.144949652"
Sep 10 14:14:44.415000 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:44.414963    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:44.515569 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:44.515532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:44.571164 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:44.571134    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:46.196387 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:46.196351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:46.403471 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:46.403442    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:52.389446 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:52.389358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:52.451533 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:52.451498    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:14:53.045128 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:14:53.045099    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bxrf8"
Sep 10 14:15:02.161381 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:02.161345    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:02.225912 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:02.225881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:30.930185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:30.930151    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:31.317963 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:31.317871    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:39.470605 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:39.470563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:39.755324 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:39.755165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:39.856984 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:39.856943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:40.261657 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:40.261611    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:40.391610 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:40.391578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:43.623416 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:43.623375    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:43.678263 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:43.678201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:50.505751 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:50.505712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:50.561568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:50.561532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:15:50.624678 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:15:50.624645    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:02.103938 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:02.103893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:24.758556 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:24.758479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:26.296837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:26.296796    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:46.162895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:46.162854    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:46.218370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:46.218338    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:48.990864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:48.990831    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:49.122685 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:49.122643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:49.223728 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:49.223700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:49.724401 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:49.724370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:49.782768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:49.782739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:50.479240 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:50.479192    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:50.534722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:50.534693    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:58.062290 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:58.062256    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:58.110692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:58.110665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:58.276296 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:58.276266    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:16:58.277515 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:58.277490    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:16:58.288353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:58.288334    2576 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 14:16:59.787461 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:59.787422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:59.890380 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:59.890348    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:16:59.990806 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:16:59.990769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:17:07.401220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:17:07.401179    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:17:07.499056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:17:07.499024    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:17:07.598953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:17:07.598920    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:17:52.720855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:17:52.720775    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:17:52.819177 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:17:52.819133    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:14.299290 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:14.299257    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:16.086560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:16.086519    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:17.910292 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:17.910255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:18.018026 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:18.017990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:19.876681 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:19.876654    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:19.976250 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:19.976198    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:20.376955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:20.376921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:20.481264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:20.481220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:21.952727 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:21.952689    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:22.269929 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:22.269859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:24.901275 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:24.901223    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:25.001607 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:25.001571    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:28.284399 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:28.284362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:28.388179 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:28.388140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:30.481157 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:30.481126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:32.580335 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:32.580303    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:32.679950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:32.679921    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:34.416861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.416824    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"]
Sep 10 14:18:34.418946 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.418930    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.421195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.421172    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 14:18:34.422210 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.422187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-w9wj4\""
Sep 10 14:18:34.422341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.422187    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:18:34.429256 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.429218    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"]
Sep 10 14:18:34.581848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.581808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0662801c-8dd3-4ffa-9ef2-4d3a556e8a83-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jtsvr\" (UID: \"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.582047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.582020    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd8km\" (UniqueName: \"kubernetes.io/projected/0662801c-8dd3-4ffa-9ef2-4d3a556e8a83-kube-api-access-qd8km\") pod \"openshift-lws-operator-fd8ccff4c-jtsvr\" (UID: \"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.682402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.682376    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0662801c-8dd3-4ffa-9ef2-4d3a556e8a83-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jtsvr\" (UID: \"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.682568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.682430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd8km\" (UniqueName: \"kubernetes.io/projected/0662801c-8dd3-4ffa-9ef2-4d3a556e8a83-kube-api-access-qd8km\") pod \"openshift-lws-operator-fd8ccff4c-jtsvr\" (UID: \"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.682784 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.682762    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0662801c-8dd3-4ffa-9ef2-4d3a556e8a83-tmp\") pod \"openshift-lws-operator-fd8ccff4c-jtsvr\" (UID: \"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.690810 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.690787    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd8km\" (UniqueName: \"kubernetes.io/projected/0662801c-8dd3-4ffa-9ef2-4d3a556e8a83-kube-api-access-qd8km\") pod \"openshift-lws-operator-fd8ccff4c-jtsvr\" (UID: \"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.728998 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.728960    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"
Sep 10 14:18:34.994372 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.994300    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr"]
Sep 10 14:18:34.997257 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:18:34.997215    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0662801c_8dd3_4ffa_9ef2_4d3a556e8a83.slice/crio-124758cb78ed32986fdcdcf78ee3ceed66f65ad49280a696ae37240ceee51c3e WatchSource:0}: Error finding container 124758cb78ed32986fdcdcf78ee3ceed66f65ad49280a696ae37240ceee51c3e: Status 404 returned error can't find the container with id 124758cb78ed32986fdcdcf78ee3ceed66f65ad49280a696ae37240ceee51c3e
Sep 10 14:18:34.998779 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:34.998763    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:18:35.831328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:35.831294    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr" event={"ID":"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83","Type":"ContainerStarted","Data":"124758cb78ed32986fdcdcf78ee3ceed66f65ad49280a696ae37240ceee51c3e"}
Sep 10 14:18:37.840314 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:37.840280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr" event={"ID":"0662801c-8dd3-4ffa-9ef2-4d3a556e8a83","Type":"ContainerStarted","Data":"3704ef1763b236135aecdd795f93cb2a40529178ac2e236f5f70da59141afddf"}
Sep 10 14:18:37.857870 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:37.857821    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-jtsvr" podStartSLOduration=1.542098714 podStartE2EDuration="3.857806481s" totalImagesPullingTime="2.315707767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:18:37.856511265 +0000 UTC m=+399.944968835" watchObservedRunningTime="2026-09-10 14:18:37.857806481 +0000 UTC m=+399.946264052"
Sep 10 14:18:38.514792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:38.514758    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:38.615818 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:38.615787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:40.460125 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:40.460099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:50.691226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:50.691172    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:50.819006 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:50.818957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:54.145512 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:54.145481    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:54.235556 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:54.235525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:54.741403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:54.741370    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.042008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.041929    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.135045 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.135006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.176130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.176104    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"]
Sep 10 14:18:55.181287 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.181266    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.183870 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.183850    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 14:18:55.183987 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.183951    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 14:18:55.184732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.184714    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 14:18:55.184732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.184724    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 14:18:55.184854 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.184754    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4t9x5\""
Sep 10 14:18:55.193305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.193283    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"]
Sep 10 14:18:55.229741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.229706    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/870bd46a-42b6-4a36-a639-f948e19bc0aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.229890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.229750    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/870bd46a-42b6-4a36-a639-f948e19bc0aa-kube-api-access-jdwb8\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.229890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.229781    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/870bd46a-42b6-4a36-a639-f948e19bc0aa-manifests\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.229890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.229808    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/870bd46a-42b6-4a36-a639-f948e19bc0aa-tmp\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.229890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.229834    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/870bd46a-42b6-4a36-a639-f948e19bc0aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.331027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.330947    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/870bd46a-42b6-4a36-a639-f948e19bc0aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.331027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.331001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/870bd46a-42b6-4a36-a639-f948e19bc0aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.331226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.331029    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/870bd46a-42b6-4a36-a639-f948e19bc0aa-kube-api-access-jdwb8\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.331226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.331049    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/870bd46a-42b6-4a36-a639-f948e19bc0aa-manifests\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.331226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.331069    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/870bd46a-42b6-4a36-a639-f948e19bc0aa-tmp\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.331635 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.331606    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/870bd46a-42b6-4a36-a639-f948e19bc0aa-manifests\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.333581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.333548    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/870bd46a-42b6-4a36-a639-f948e19bc0aa-tmp\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.333687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.333642    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/870bd46a-42b6-4a36-a639-f948e19bc0aa-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.333739 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.333698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/870bd46a-42b6-4a36-a639-f948e19bc0aa-webhook-cert\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.346941 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.346915    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdwb8\" (UniqueName: \"kubernetes.io/projected/870bd46a-42b6-4a36-a639-f948e19bc0aa-kube-api-access-jdwb8\") pod \"opendatahub-operator-controller-manager-55f59f54dd-qgt9h\" (UID: \"870bd46a-42b6-4a36-a639-f948e19bc0aa\") " pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.366344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.366321    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.467887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.467848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.491967 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.491947    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:18:55.569037 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.569000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.669503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.669392    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.770800 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.770769    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"]
Sep 10 14:18:55.774575 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:18:55.774548    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod870bd46a_42b6_4a36_a639_f948e19bc0aa.slice/crio-c29236f29cbcaea6ae37d548cbfe24ff766e33419c61ce32b8cfc9f33e08b6d5 WatchSource:0}: Error finding container c29236f29cbcaea6ae37d548cbfe24ff766e33419c61ce32b8cfc9f33e08b6d5: Status 404 returned error can't find the container with id c29236f29cbcaea6ae37d548cbfe24ff766e33419c61ce32b8cfc9f33e08b6d5
Sep 10 14:18:55.776449 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.776428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:18:55.905532 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:55.905503    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h" event={"ID":"870bd46a-42b6-4a36-a639-f948e19bc0aa","Type":"ContainerStarted","Data":"c29236f29cbcaea6ae37d548cbfe24ff766e33419c61ce32b8cfc9f33e08b6d5"}
Sep 10 14:18:58.919744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:18:58.919718    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h" event={"ID":"870bd46a-42b6-4a36-a639-f948e19bc0aa","Type":"ContainerStarted","Data":"ad4935dc21e501572e175a163ceed34541db9497beb6cb93342aaff12f3df0aa"}
Sep 10 14:19:00.928062 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:00.928032    2576 generic.go:356] "Generic (PLEG): container finished" podID="870bd46a-42b6-4a36-a639-f948e19bc0aa" containerID="ad4935dc21e501572e175a163ceed34541db9497beb6cb93342aaff12f3df0aa" exitCode=0
Sep 10 14:19:00.928462 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:00.928116    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h" event={"ID":"870bd46a-42b6-4a36-a639-f948e19bc0aa","Type":"ContainerDied","Data":"ad4935dc21e501572e175a163ceed34541db9497beb6cb93342aaff12f3df0aa"}
Sep 10 14:19:01.934054 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:01.934019    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h" event={"ID":"870bd46a-42b6-4a36-a639-f948e19bc0aa","Type":"ContainerStarted","Data":"26c84035c6ac4c32ba2fef7abe5f0d056e0172ae2e41e83d8eb00bfe40504cb9"}
Sep 10 14:19:01.934500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:01.934269    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:19:01.955316 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:01.955267    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h" podStartSLOduration=2.383951659 podStartE2EDuration="6.955253039s" totalImagesPullingTime="3.526742089s" totalInitContainerRuntime="1.044559291s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:18:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:01.953857164 +0000 UTC m=+424.042314735" watchObservedRunningTime="2026-09-10 14:19:01.955253039 +0000 UTC m=+424.043710600"
Sep 10 14:19:07.382856 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:07.382819    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:07.476420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:07.476391    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:12.941201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:12.941170    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55f59f54dd-qgt9h"
Sep 10 14:19:14.504128 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:14.504093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:14.615173 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:14.615142    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:16.294509 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.294479    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"]
Sep 10 14:19:16.303270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.303225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.304566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.304535    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:16.305597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.305581    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 14:19:16.305833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.305818    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 14:19:16.305912 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.305894    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 14:19:16.306740 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.306701    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 14:19:16.310161 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.310140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"]
Sep 10 14:19:16.394673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.394632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-tls-certs\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.394846 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.394762    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wjnf\" (UniqueName: \"kubernetes.io/projected/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-kube-api-access-4wjnf\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.394846 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.394797    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-tmp\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.486185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.486140    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:16.495271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.495246    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-tls-certs\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.495395 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.495314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wjnf\" (UniqueName: \"kubernetes.io/projected/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-kube-api-access-4wjnf\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.495395 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.495336    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-tmp\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.497589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.497572    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-tmp\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.497934 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.497918    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-tls-certs\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.502837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.502819    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wjnf\" (UniqueName: \"kubernetes.io/projected/fd18ebe1-c2ac-4a4a-8564-836411fd2a2a-kube-api-access-4wjnf\") pod \"kube-auth-proxy-85dcd64b9c-tgw8z\" (UID: \"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a\") " pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.619330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.619221    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"
Sep 10 14:19:16.894741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.894623    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z"]
Sep 10 14:19:16.897475 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:19:16.897449    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd18ebe1_c2ac_4a4a_8564_836411fd2a2a.slice/crio-0f5c5f0f673b80f3f764fd67d6d48a84b20e6a1b93e3ece9da795aef024b43ed WatchSource:0}: Error finding container 0f5c5f0f673b80f3f764fd67d6d48a84b20e6a1b93e3ece9da795aef024b43ed: Status 404 returned error can't find the container with id 0f5c5f0f673b80f3f764fd67d6d48a84b20e6a1b93e3ece9da795aef024b43ed
Sep 10 14:19:16.992727 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:16.992688    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z" event={"ID":"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a","Type":"ContainerStarted","Data":"0f5c5f0f673b80f3f764fd67d6d48a84b20e6a1b93e3ece9da795aef024b43ed"}
Sep 10 14:19:17.538111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.538079    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"]
Sep 10 14:19:17.542478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.542462    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.544968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.544941    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2bd7v\""
Sep 10 14:19:17.545086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.544969    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 14:19:17.545086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.544983    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:19:17.545086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.545010    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 14:19:17.545086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.544989    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 14:19:17.553600 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.553580    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"]
Sep 10 14:19:17.558315 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.558289    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:17.603716 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603687    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.603879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603723    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjbq9\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-kube-api-access-zjbq9\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.603879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603754    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.603879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603823    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.603879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603851    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.603879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603876    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.604039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.603896    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.626185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.626149    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:17.705305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705263    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.705499 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705314    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjbq9\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-kube-api-access-zjbq9\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.705499 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705367    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.705499 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705427    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.705499 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705464    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.705730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705501    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.705730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.705529    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.706223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.706159    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.708722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.708677    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.709417 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.709363    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.710035 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.710009    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.711266 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.711185    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.713906 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.713863    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjbq9\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-kube-api-access-zjbq9\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.714198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.714173    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-kw95x\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:17.852588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:17.852504    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:18.143523 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:18.143395    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"]
Sep 10 14:19:18.148192 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:19:18.148152    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4ccb343_3214_4ea3_9f9b_a8f82b70f2c3.slice/crio-168c6200162e28c9056abd659290ffb3a04f1c892e9ad126a82b8c7190be839c WatchSource:0}: Error finding container 168c6200162e28c9056abd659290ffb3a04f1c892e9ad126a82b8c7190be839c: Status 404 returned error can't find the container with id 168c6200162e28c9056abd659290ffb3a04f1c892e9ad126a82b8c7190be839c
Sep 10 14:19:18.608024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:18.607988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:19.007721 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:19.007632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" event={"ID":"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3","Type":"ContainerStarted","Data":"168c6200162e28c9056abd659290ffb3a04f1c892e9ad126a82b8c7190be839c"}
Sep 10 14:19:20.015025 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:20.014935    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z" event={"ID":"fd18ebe1-c2ac-4a4a-8564-836411fd2a2a","Type":"ContainerStarted","Data":"19dd9b9959196f2a7b6b64b5313906b8bfbe01f5b23ef1e41e78d70f36b224f1"}
Sep 10 14:19:20.033404 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:20.033191    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85dcd64b9c-tgw8z" podStartSLOduration=1.184651908 podStartE2EDuration="4.03316823s" totalImagesPullingTime="2.848516322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:20.030758664 +0000 UTC m=+442.119216235" watchObservedRunningTime="2026-09-10 14:19:20.03316823 +0000 UTC m=+442.121625832"
Sep 10 14:19:21.532950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:21.532909    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:19:21.533248 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:21.532978    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:19:22.025295 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.025257    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" event={"ID":"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3","Type":"ContainerStarted","Data":"29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9"}
Sep 10 14:19:22.025455 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.025413    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:22.027095 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.027068    2576 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-kw95x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 14:19:22.027213 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.027114    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" podUID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:19:22.048580 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.048531    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" podStartSLOduration=1.666959243 podStartE2EDuration="5.048514979s" totalImagesPullingTime="3.381555736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:22.046416923 +0000 UTC m=+444.134874490" watchObservedRunningTime="2026-09-10 14:19:22.048514979 +0000 UTC m=+444.136972550"
Sep 10 14:19:22.286785 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.286702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:22.389527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.389495    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:22.885388 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:22.885349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:23.004634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:23.004602    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:23.029989 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:23.029964    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:23.105704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:23.105676    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:25.553272 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:25.553225    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:25.743992 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:25.743960    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:25.846673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:25.846594    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:26.023995 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.023962    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dj2ng"]
Sep 10 14:19:26.027532 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.027515    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.030135 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.030113    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 10 14:19:26.038125 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.038100    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dj2ng"]
Sep 10 14:19:26.057296 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.057271    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:26.073409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.073385    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crz5k\" (UniqueName: \"kubernetes.io/projected/90ece399-7522-4544-8c12-60067c925e44-kube-api-access-crz5k\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.073513 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.073438    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ece399-7522-4544-8c12-60067c925e44-tls-certs\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.174021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.173994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crz5k\" (UniqueName: \"kubernetes.io/projected/90ece399-7522-4544-8c12-60067c925e44-kube-api-access-crz5k\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.174182 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.174047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ece399-7522-4544-8c12-60067c925e44-tls-certs\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.174182 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:19:26.174159    2576 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Sep 10 14:19:26.174306 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:19:26.174220    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90ece399-7522-4544-8c12-60067c925e44-tls-certs podName:90ece399-7522-4544-8c12-60067c925e44 nodeName:}" failed. No retries permitted until 2026-09-10 14:19:26.674201886 +0000 UTC m=+448.762659435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/90ece399-7522-4544-8c12-60067c925e44-tls-certs") pod "model-serving-api-54d89bd5fc-dj2ng" (UID: "90ece399-7522-4544-8c12-60067c925e44") : secret "model-serving-api-tls" not found
Sep 10 14:19:26.184720 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.184695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crz5k\" (UniqueName: \"kubernetes.io/projected/90ece399-7522-4544-8c12-60067c925e44-kube-api-access-crz5k\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.362124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.362083    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:26.678546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.678513    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ece399-7522-4544-8c12-60067c925e44-tls-certs\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.681145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.681118    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/90ece399-7522-4544-8c12-60067c925e44-tls-certs\") pod \"model-serving-api-54d89bd5fc-dj2ng\" (UID: \"90ece399-7522-4544-8c12-60067c925e44\") " pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:26.938890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:26.938794    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:27.216517 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:27.216314    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-dj2ng"]
Sep 10 14:19:27.219453 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:19:27.219412    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90ece399_7522_4544_8c12_60067c925e44.slice/crio-3e3a8c8680632c11cca5808f172645c4bee3ab90354ad99f84455a6ed485bb6e WatchSource:0}: Error finding container 3e3a8c8680632c11cca5808f172645c4bee3ab90354ad99f84455a6ed485bb6e: Status 404 returned error can't find the container with id 3e3a8c8680632c11cca5808f172645c4bee3ab90354ad99f84455a6ed485bb6e
Sep 10 14:19:28.051504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:28.051465    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng" event={"ID":"90ece399-7522-4544-8c12-60067c925e44","Type":"ContainerStarted","Data":"3e3a8c8680632c11cca5808f172645c4bee3ab90354ad99f84455a6ed485bb6e"}
Sep 10 14:19:29.056778 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:29.056744    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng" event={"ID":"90ece399-7522-4544-8c12-60067c925e44","Type":"ContainerStarted","Data":"ce99a636b3ff883057272b4b70b9f79bc3b6b68d52951d1d6d60706f9b75a9d9"}
Sep 10 14:19:29.057117 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:29.056888    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:19:29.072922 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:29.072883    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng" podStartSLOduration=1.338155003 podStartE2EDuration="3.072869933s" totalImagesPullingTime="1.73471493s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:29.07113945 +0000 UTC m=+451.159597023" watchObservedRunningTime="2026-09-10 14:19:29.072869933 +0000 UTC m=+451.161327543"
Sep 10 14:19:34.536876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.536839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:34.602295 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.602263    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"]
Sep 10 14:19:34.610597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.610569    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.613272 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.613224    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-rl6v9\""
Sep 10 14:19:34.613469 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.613445    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"]
Sep 10 14:19:34.613543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.613492    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 14:19:34.616089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.616059    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.617733 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.617711    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"]
Sep 10 14:19:34.618417 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.618395    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 14:19:34.618528 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.618437    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ntqqd\""
Sep 10 14:19:34.623497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.623468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:34.631689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.631666    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"]
Sep 10 14:19:34.730880 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.730853    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:34.745202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.745163    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243380c0-5722-40b2-bf45-727be8411511-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mx5dx\" (UID: \"243380c0-5722-40b2-bf45-727be8411511\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.745397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.745345    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnv74\" (UniqueName: \"kubernetes.io/projected/74b7d555-ab98-434a-b5b7-95688e34645a-kube-api-access-pnv74\") pod \"kserve-controller-manager-d6f4b65b8-qw6zk\" (UID: \"74b7d555-ab98-434a-b5b7-95688e34645a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.745470 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.745437    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b7d555-ab98-434a-b5b7-95688e34645a-cert\") pod \"kserve-controller-manager-d6f4b65b8-qw6zk\" (UID: \"74b7d555-ab98-434a-b5b7-95688e34645a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.745525 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.745470    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zf5gk\" (UniqueName: \"kubernetes.io/projected/243380c0-5722-40b2-bf45-727be8411511-kube-api-access-zf5gk\") pod \"llmisvc-controller-manager-777c6bd7d5-mx5dx\" (UID: \"243380c0-5722-40b2-bf45-727be8411511\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.846022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.845910    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:34.846022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.845969    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b7d555-ab98-434a-b5b7-95688e34645a-cert\") pod \"kserve-controller-manager-d6f4b65b8-qw6zk\" (UID: \"74b7d555-ab98-434a-b5b7-95688e34645a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.846022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.846015    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zf5gk\" (UniqueName: \"kubernetes.io/projected/243380c0-5722-40b2-bf45-727be8411511-kube-api-access-zf5gk\") pod \"llmisvc-controller-manager-777c6bd7d5-mx5dx\" (UID: \"243380c0-5722-40b2-bf45-727be8411511\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.846330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.846072    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243380c0-5722-40b2-bf45-727be8411511-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mx5dx\" (UID: \"243380c0-5722-40b2-bf45-727be8411511\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.846330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.846119    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnv74\" (UniqueName: \"kubernetes.io/projected/74b7d555-ab98-434a-b5b7-95688e34645a-kube-api-access-pnv74\") pod \"kserve-controller-manager-d6f4b65b8-qw6zk\" (UID: \"74b7d555-ab98-434a-b5b7-95688e34645a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.848847 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.848818    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/243380c0-5722-40b2-bf45-727be8411511-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mx5dx\" (UID: \"243380c0-5722-40b2-bf45-727be8411511\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.848962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.848825    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/74b7d555-ab98-434a-b5b7-95688e34645a-cert\") pod \"kserve-controller-manager-d6f4b65b8-qw6zk\" (UID: \"74b7d555-ab98-434a-b5b7-95688e34645a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.863707 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.863674    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zf5gk\" (UniqueName: \"kubernetes.io/projected/243380c0-5722-40b2-bf45-727be8411511-kube-api-access-zf5gk\") pod \"llmisvc-controller-manager-777c6bd7d5-mx5dx\" (UID: \"243380c0-5722-40b2-bf45-727be8411511\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.876631 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.876604    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnv74\" (UniqueName: \"kubernetes.io/projected/74b7d555-ab98-434a-b5b7-95688e34645a-kube-api-access-pnv74\") pod \"kserve-controller-manager-d6f4b65b8-qw6zk\" (UID: \"74b7d555-ab98-434a-b5b7-95688e34645a\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.925320 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.925284    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:34.931637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.931617    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:34.995372 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:34.995296    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:19:35.202556 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:35.202526    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"]
Sep 10 14:19:35.207310 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:35.206666    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"]
Sep 10 14:19:35.208470 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:19:35.208380    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74b7d555_ab98_434a_b5b7_95688e34645a.slice/crio-61c380cb72ce17d923e5b60fd914229a88bcde3bc5cd34ec00f7ef043e00d92c WatchSource:0}: Error finding container 61c380cb72ce17d923e5b60fd914229a88bcde3bc5cd34ec00f7ef043e00d92c: Status 404 returned error can't find the container with id 61c380cb72ce17d923e5b60fd914229a88bcde3bc5cd34ec00f7ef043e00d92c
Sep 10 14:19:35.211804 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:19:35.211782    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod243380c0_5722_40b2_bf45_727be8411511.slice/crio-c511ffa7277949bf4057a9895ea3512c7a2f626ecb58f87be03cb7ee56ff2447 WatchSource:0}: Error finding container c511ffa7277949bf4057a9895ea3512c7a2f626ecb58f87be03cb7ee56ff2447: Status 404 returned error can't find the container with id c511ffa7277949bf4057a9895ea3512c7a2f626ecb58f87be03cb7ee56ff2447
Sep 10 14:19:36.087968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.087883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx" event={"ID":"243380c0-5722-40b2-bf45-727be8411511","Type":"ContainerStarted","Data":"c511ffa7277949bf4057a9895ea3512c7a2f626ecb58f87be03cb7ee56ff2447"}
Sep 10 14:19:36.089549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.089512    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk" event={"ID":"74b7d555-ab98-434a-b5b7-95688e34645a","Type":"ContainerStarted","Data":"61c380cb72ce17d923e5b60fd914229a88bcde3bc5cd34ec00f7ef043e00d92c"}
Sep 10 14:19:36.465537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.465503    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"]
Sep 10 14:19:36.466005 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.465751    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" podUID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" containerName="discovery" containerID="cri-o://29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9" gracePeriod=30
Sep 10 14:19:36.733527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.733501    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:36.864115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjbq9\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-kube-api-access-zjbq9\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.864321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864122    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-token\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.864321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864150    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-ca-configmap\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.864321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864177    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-dns-cert\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.864321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864197    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-local-certs\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.864321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864283    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-kubeconfig\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.864321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864310    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-cacerts\") pod \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\" (UID: \"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3\") "
Sep 10 14:19:36.865226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.864965    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:19:36.867643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.867614    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-local-certs" (OuterVolumeSpecName: "local-certs") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:19:36.869252 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.869180    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:19:36.869252 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.869208    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-token" (OuterVolumeSpecName: "istio-token") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:19:36.869557 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.869530    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-kube-api-access-zjbq9" (OuterVolumeSpecName: "kube-api-access-zjbq9") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "kube-api-access-zjbq9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:19:36.869808 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.869779    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-cacerts" (OuterVolumeSpecName: "cacerts") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:19:36.869914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.869874    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" (UID: "f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:19:36.965875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965791    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:36.965875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965824    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-ca-configmap\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:36.965875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965842    2576 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-csr-dns-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:36.965875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965856    2576 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-local-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:36.965875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965871    2576 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-istio-kubeconfig\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:36.965875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965881    2576 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-cacerts\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:36.966385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:36.965891    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjbq9\" (UniqueName: \"kubernetes.io/projected/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3-kube-api-access-zjbq9\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:19:37.096148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.096064    2576 generic.go:356] "Generic (PLEG): container finished" podID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" containerID="29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9" exitCode=0
Sep 10 14:19:37.096148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.096133    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"
Sep 10 14:19:37.096672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.096156    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" event={"ID":"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3","Type":"ContainerDied","Data":"29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9"}
Sep 10 14:19:37.096672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.096221    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x" event={"ID":"f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3","Type":"ContainerDied","Data":"168c6200162e28c9056abd659290ffb3a04f1c892e9ad126a82b8c7190be839c"}
Sep 10 14:19:37.096672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.096281    2576 scope.go:117] "RemoveContainer" containerID="29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9"
Sep 10 14:19:37.125785 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.125757    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"]
Sep 10 14:19:37.135730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.135671    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-kw95x"]
Sep 10 14:19:37.319509 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.319479    2576 scope.go:117] "RemoveContainer" containerID="29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9"
Sep 10 14:19:37.319926 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:19:37.319900    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9\": container with ID starting with 29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9 not found: ID does not exist" containerID="29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9"
Sep 10 14:19:37.320009 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:37.319935    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9"} err="failed to get container status \"29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9\": rpc error: code = NotFound desc = could not find container \"29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9\": container with ID starting with 29ffa3e61f445e212c5918cc93eacfbe2f6b4ec0ee4c5f081a192609a6979ad9 not found: ID does not exist"
Sep 10 14:19:38.102136 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:38.102089    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk" event={"ID":"74b7d555-ab98-434a-b5b7-95688e34645a","Type":"ContainerStarted","Data":"b1919f66b3ffa3a1dfbb65c1899a1c17a58782f3010ccc2f6c274c4ace16e407"}
Sep 10 14:19:38.102574 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:38.102161    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:19:38.123753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:38.123704    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk" podStartSLOduration=1.966880365 podStartE2EDuration="4.123682712s" totalImagesPullingTime="2.156802347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:38.121382658 +0000 UTC m=+460.209840242" watchObservedRunningTime="2026-09-10 14:19:38.123682712 +0000 UTC m=+460.212140284"
Sep 10 14:19:38.426448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:38.426418    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" path="/var/lib/kubelet/pods/f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3/volumes"
Sep 10 14:19:39.108833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:39.108795    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx" event={"ID":"243380c0-5722-40b2-bf45-727be8411511","Type":"ContainerStarted","Data":"2b72e52ebbc4da7e07e4fafae7889930de9da2a2f22af0daa864642c34b35979"}
Sep 10 14:19:39.109169 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:39.108862    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:19:39.128710 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:39.128661    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx" podStartSLOduration=2.2245668690000002 podStartE2EDuration="5.128645612s" totalImagesPullingTime="2.904078743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:39.127071483 +0000 UTC m=+461.215529081" watchObservedRunningTime="2026-09-10 14:19:39.128645612 +0000 UTC m=+461.217103182"
Sep 10 14:19:40.066071 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:19:40.066042    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-dj2ng"
Sep 10 14:20:09.114276 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:09.114244    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-qw6zk"
Sep 10 14:20:10.115387 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:10.115356    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mx5dx"
Sep 10 14:20:26.595190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:26.595148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:26.683766 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:26.683740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:51.551085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:51.551008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:52.357572 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:52.357542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:53.148991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:53.148957    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:55.914742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.914709    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"]
Sep 10 14:20:55.915108 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.915039    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" containerName="discovery"
Sep 10 14:20:55.915108 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.915053    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" containerName="discovery"
Sep 10 14:20:55.915185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.915123    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4ccb343-3214-4ea3-9f9b-a8f82b70f2c3" containerName="discovery"
Sep 10 14:20:55.917670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.917650    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:55.920266 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.920248    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j42sg\""
Sep 10 14:20:55.920412 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.920394    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 14:20:55.920492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.920413    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 14:20:55.930045 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.930025    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"]
Sep 10 14:20:55.986163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.986131    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96a636d9-d2d3-4781-844c-351e4573e25c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:55.986320 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:55.986192    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssnns\" (UniqueName: \"kubernetes.io/projected/96a636d9-d2d3-4781-844c-351e4573e25c-kube-api-access-ssnns\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:56.086928 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.086893    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96a636d9-d2d3-4781-844c-351e4573e25c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:56.087089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.086968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssnns\" (UniqueName: \"kubernetes.io/projected/96a636d9-d2d3-4781-844c-351e4573e25c-kube-api-access-ssnns\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:56.087300 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.087279    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96a636d9-d2d3-4781-844c-351e4573e25c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:56.098176 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.098151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssnns\" (UniqueName: \"kubernetes.io/projected/96a636d9-d2d3-4781-844c-351e4573e25c-kube-api-access-ssnns\") pod \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:56.228386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.228356    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:20:56.426868 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.426829    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:56.528206 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.526997    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"]
Sep 10 14:20:56.530389 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:20:56.530359    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a636d9_d2d3_4781_844c_351e4573e25c.slice/crio-483bb3312349b520d0a08361c1152002380db8315eac95ca9072faa3e7e22c32 WatchSource:0}: Error finding container 483bb3312349b520d0a08361c1152002380db8315eac95ca9072faa3e7e22c32: Status 404 returned error can't find the container with id 483bb3312349b520d0a08361c1152002380db8315eac95ca9072faa3e7e22c32
Sep 10 14:20:56.559491 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.559461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:56.677589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:56.677559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:57.416987 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.416951    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc" event={"ID":"96a636d9-d2d3-4781-844c-351e4573e25c","Type":"ContainerStarted","Data":"483bb3312349b520d0a08361c1152002380db8315eac95ca9072faa3e7e22c32"}
Sep 10 14:20:57.838506 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.838472    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"]
Sep 10 14:20:57.841896 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.841853    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:20:57.844823 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.844798    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 14:20:57.844955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.844807    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-w78jf\""
Sep 10 14:20:57.852617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.852590    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"]
Sep 10 14:20:57.898954 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.898917    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9p2n\" (UniqueName: \"kubernetes.io/projected/c846bd41-86da-4f04-a5f9-7aedf9f95e0a-kube-api-access-p9p2n\") pod \"dns-operator-controller-manager-7b84cd7c9c-mt5wl\" (UID: \"c846bd41-86da-4f04-a5f9-7aedf9f95e0a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:20:57.999426 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:57.999386    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9p2n\" (UniqueName: \"kubernetes.io/projected/c846bd41-86da-4f04-a5f9-7aedf9f95e0a-kube-api-access-p9p2n\") pod \"dns-operator-controller-manager-7b84cd7c9c-mt5wl\" (UID: \"c846bd41-86da-4f04-a5f9-7aedf9f95e0a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:20:58.009814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:58.009786    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9p2n\" (UniqueName: \"kubernetes.io/projected/c846bd41-86da-4f04-a5f9-7aedf9f95e0a-kube-api-access-p9p2n\") pod \"dns-operator-controller-manager-7b84cd7c9c-mt5wl\" (UID: \"c846bd41-86da-4f04-a5f9-7aedf9f95e0a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:20:58.156725 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:58.156636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:20:58.387889 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:58.387756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:58.550208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:58.550173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:58.811049 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:58.810941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:59.223227 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:59.223187    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"]
Sep 10 14:20:59.226825 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:20:59.226795    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc846bd41_86da_4f04_a5f9_7aedf9f95e0a.slice/crio-4e0bdcde59dfb925104fbd13f13bd635acbd46eea22bbf70f55dc98cc0645916 WatchSource:0}: Error finding container 4e0bdcde59dfb925104fbd13f13bd635acbd46eea22bbf70f55dc98cc0645916: Status 404 returned error can't find the container with id 4e0bdcde59dfb925104fbd13f13bd635acbd46eea22bbf70f55dc98cc0645916
Sep 10 14:20:59.428756 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:20:59.428710    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl" event={"ID":"c846bd41-86da-4f04-a5f9-7aedf9f95e0a","Type":"ContainerStarted","Data":"4e0bdcde59dfb925104fbd13f13bd635acbd46eea22bbf70f55dc98cc0645916"}
Sep 10 14:21:00.664596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.664562    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-p5n2j"]
Sep 10 14:21:00.668402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.668375    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:00.671156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.671133    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-6z9v2\""
Sep 10 14:21:00.690334 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.690275    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-p5n2j"]
Sep 10 14:21:00.717193 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.717153    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lmcm\" (UniqueName: \"kubernetes.io/projected/e361ffe9-56a4-4a89-8f2e-fa37702ae7ae-kube-api-access-4lmcm\") pod \"authorino-operator-76b478bd78-p5n2j\" (UID: \"e361ffe9-56a4-4a89-8f2e-fa37702ae7ae\") " pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:00.817877 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.817835    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmcm\" (UniqueName: \"kubernetes.io/projected/e361ffe9-56a4-4a89-8f2e-fa37702ae7ae-kube-api-access-4lmcm\") pod \"authorino-operator-76b478bd78-p5n2j\" (UID: \"e361ffe9-56a4-4a89-8f2e-fa37702ae7ae\") " pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:00.830946 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.830914    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lmcm\" (UniqueName: \"kubernetes.io/projected/e361ffe9-56a4-4a89-8f2e-fa37702ae7ae-kube-api-access-4lmcm\") pod \"authorino-operator-76b478bd78-p5n2j\" (UID: \"e361ffe9-56a4-4a89-8f2e-fa37702ae7ae\") " pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:00.980937 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:00.980894    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:01.175132 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.175101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:01.402018 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.401962    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-p5n2j"]
Sep 10 14:21:01.406438 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:21:01.406407    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode361ffe9_56a4_4a89_8f2e_fa37702ae7ae.slice/crio-c8090f9a2fc7859659e803de155da6edbedf14f98d28032d6c7b4fa18bedda83 WatchSource:0}: Error finding container c8090f9a2fc7859659e803de155da6edbedf14f98d28032d6c7b4fa18bedda83: Status 404 returned error can't find the container with id c8090f9a2fc7859659e803de155da6edbedf14f98d28032d6c7b4fa18bedda83
Sep 10 14:21:01.439363 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.439331    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc" event={"ID":"96a636d9-d2d3-4781-844c-351e4573e25c","Type":"ContainerStarted","Data":"5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31"}
Sep 10 14:21:01.439641 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.439596    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:01.440800 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.440777    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j" event={"ID":"e361ffe9-56a4-4a89-8f2e-fa37702ae7ae","Type":"ContainerStarted","Data":"c8090f9a2fc7859659e803de155da6edbedf14f98d28032d6c7b4fa18bedda83"}
Sep 10 14:21:01.462382 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.462296    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc" podStartSLOduration=1.789431046 podStartE2EDuration="6.462277701s" totalImagesPullingTime="4.672846655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:20:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:01.461865515 +0000 UTC m=+543.550323089" watchObservedRunningTime="2026-09-10 14:21:01.462277701 +0000 UTC m=+543.550735274"
Sep 10 14:21:01.837584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:01.837538    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:02.000992 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.000955    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:02.116373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.116324    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 14:21:02.446104 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.446067    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl" event={"ID":"c846bd41-86da-4f04-a5f9-7aedf9f95e0a","Type":"ContainerStarted","Data":"f0d8d61751dacb46fbf3dee09b249cede022adf7223f541efa397f9a8f095462"}
Sep 10 14:21:02.446386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.446366    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:21:02.482081 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.477974    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl" podStartSLOduration=2.594653568 podStartE2EDuration="5.477953426s" totalImagesPullingTime="2.883299858s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:20:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:02.476818084 +0000 UTC m=+544.565275655" watchObservedRunningTime="2026-09-10 14:21:02.477953426 +0000 UTC m=+544.566410998"
Sep 10 14:21:02.775951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.775876    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"]
Sep 10 14:21:02.779377 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.779359    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:02.783869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.783843    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-v2gs9\""
Sep 10 14:21:02.790053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.790025    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"]
Sep 10 14:21:02.836831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.836795    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tmf7\" (UniqueName: \"kubernetes.io/projected/0ffdeb52-fe42-4770-98e3-d8f69c3820bc-kube-api-access-8tmf7\") pod \"limitador-operator-controller-manager-74495c69f8-pbhbr\" (UID: \"0ffdeb52-fe42-4770-98e3-d8f69c3820bc\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:02.937856 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.937809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tmf7\" (UniqueName: \"kubernetes.io/projected/0ffdeb52-fe42-4770-98e3-d8f69c3820bc-kube-api-access-8tmf7\") pod \"limitador-operator-controller-manager-74495c69f8-pbhbr\" (UID: \"0ffdeb52-fe42-4770-98e3-d8f69c3820bc\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:02.946521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:02.946492    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tmf7\" (UniqueName: \"kubernetes.io/projected/0ffdeb52-fe42-4770-98e3-d8f69c3820bc-kube-api-access-8tmf7\") pod \"limitador-operator-controller-manager-74495c69f8-pbhbr\" (UID: \"0ffdeb52-fe42-4770-98e3-d8f69c3820bc\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:03.090074 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.089999    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:03.384384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.384285    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"]
Sep 10 14:21:03.387701 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:21:03.387672    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffdeb52_fe42_4770_98e3_d8f69c3820bc.slice/crio-628ce67f61ee47dd1d777e96364f2829ee45246e6f9cc31dd0e94fbdbee5276e WatchSource:0}: Error finding container 628ce67f61ee47dd1d777e96364f2829ee45246e6f9cc31dd0e94fbdbee5276e: Status 404 returned error can't find the container with id 628ce67f61ee47dd1d777e96364f2829ee45246e6f9cc31dd0e94fbdbee5276e
Sep 10 14:21:03.453061 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.453018    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j" event={"ID":"e361ffe9-56a4-4a89-8f2e-fa37702ae7ae","Type":"ContainerStarted","Data":"d16e3af5ed03d165c277fe6a2046da8141a90c67b06cc9be9078ce8465cb7f0b"}
Sep 10 14:21:03.453351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.453153    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:03.454774 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.454748    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr" event={"ID":"0ffdeb52-fe42-4770-98e3-d8f69c3820bc","Type":"ContainerStarted","Data":"628ce67f61ee47dd1d777e96364f2829ee45246e6f9cc31dd0e94fbdbee5276e"}
Sep 10 14:21:03.474389 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.474336    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j" podStartSLOduration=1.7773788879999999 podStartE2EDuration="3.47431903s" totalImagesPullingTime="1.696940142s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:03.473582894 +0000 UTC m=+545.562040465" watchObservedRunningTime="2026-09-10 14:21:03.47431903 +0000 UTC m=+545.562776603"
Sep 10 14:21:03.638952 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:03.638873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:04.306429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.306219    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"]
Sep 10 14:21:04.310335 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.310302    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.312962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.312730    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 14:21:04.312962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.312770    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 14:21:04.312962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.312799    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-gz7vr\""
Sep 10 14:21:04.316188 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.316141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"]
Sep 10 14:21:04.321283 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.321260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:04.350246 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.350204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee23449c-2560-4f76-96fa-57b77cd7a0e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.350402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.350319    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee23449c-2560-4f76-96fa-57b77cd7a0e0-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.350402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.350356    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdcwf\" (UniqueName: \"kubernetes.io/projected/ee23449c-2560-4f76-96fa-57b77cd7a0e0-kube-api-access-xdcwf\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.450811 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.450780    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee23449c-2560-4f76-96fa-57b77cd7a0e0-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.450992 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.450817    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdcwf\" (UniqueName: \"kubernetes.io/projected/ee23449c-2560-4f76-96fa-57b77cd7a0e0-kube-api-access-xdcwf\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.450992 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.450912    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee23449c-2560-4f76-96fa-57b77cd7a0e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.451115 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:04.451067    2576 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 10 14:21:04.451177 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:04.451129    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ee23449c-2560-4f76-96fa-57b77cd7a0e0-plugin-serving-cert podName:ee23449c-2560-4f76-96fa-57b77cd7a0e0 nodeName:}" failed. No retries permitted until 2026-09-10 14:21:04.951111082 +0000 UTC m=+547.039568636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/ee23449c-2560-4f76-96fa-57b77cd7a0e0-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-gh5th" (UID: "ee23449c-2560-4f76-96fa-57b77cd7a0e0") : secret "plugin-serving-cert" not found
Sep 10 14:21:04.451658 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.451630    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ee23449c-2560-4f76-96fa-57b77cd7a0e0-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.460263 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.460212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdcwf\" (UniqueName: \"kubernetes.io/projected/ee23449c-2560-4f76-96fa-57b77cd7a0e0-kube-api-access-xdcwf\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.664080 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.663990    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:04.956379 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.956348    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee23449c-2560-4f76-96fa-57b77cd7a0e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:04.958802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:04.958781    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ee23449c-2560-4f76-96fa-57b77cd7a0e0-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-gh5th\" (UID: \"ee23449c-2560-4f76-96fa-57b77cd7a0e0\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:05.223923 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:05.223834    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"
Sep 10 14:21:05.466066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:05.466032    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr" event={"ID":"0ffdeb52-fe42-4770-98e3-d8f69c3820bc","Type":"ContainerStarted","Data":"7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810"}
Sep 10 14:21:05.466575 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:05.466117    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:05.484670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:05.484587    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr" podStartSLOduration=1.854353782 podStartE2EDuration="3.484570405s" totalImagesPullingTime="1.630216623s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:05.483511832 +0000 UTC m=+547.571969403" watchObservedRunningTime="2026-09-10 14:21:05.484570405 +0000 UTC m=+547.573027975"
Sep 10 14:21:05.497918 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:05.497893    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th"]
Sep 10 14:21:05.501770 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:21:05.501745    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee23449c_2560_4f76_96fa_57b77cd7a0e0.slice/crio-2e7b36017f9788b8ec8040da5592a2559df9715a12833821352b43fc61a4b2b1 WatchSource:0}: Error finding container 2e7b36017f9788b8ec8040da5592a2559df9715a12833821352b43fc61a4b2b1: Status 404 returned error can't find the container with id 2e7b36017f9788b8ec8040da5592a2559df9715a12833821352b43fc61a4b2b1
Sep 10 14:21:06.474804 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:06.474743    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th" event={"ID":"ee23449c-2560-4f76-96fa-57b77cd7a0e0","Type":"ContainerStarted","Data":"2e7b36017f9788b8ec8040da5592a2559df9715a12833821352b43fc61a4b2b1"}
Sep 10 14:21:06.782600 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:06.782515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:06.861715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:06.861677    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:07.172483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:07.172451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:07.245151 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:07.245114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:07.580620 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:07.580542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:07.682610 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:07.682580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:10.496706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:10.496659    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th" event={"ID":"ee23449c-2560-4f76-96fa-57b77cd7a0e0","Type":"ContainerStarted","Data":"7af2734ea66350376460a34ce7f4624e153e902ccf79804328540c2324930c13"}
Sep 10 14:21:10.522933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:10.522880    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-gh5th" podStartSLOduration=1.9547938519999999 podStartE2EDuration="6.522865505s" totalImagesPullingTime="4.568071653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:10.520825714 +0000 UTC m=+552.609283285" watchObservedRunningTime="2026-09-10 14:21:10.522865505 +0000 UTC m=+552.611323075"
Sep 10 14:21:12.448542 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:12.448510    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:13.457649 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:13.457617    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-mt5wl"
Sep 10 14:21:13.557062 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:13.557028    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:13.653975 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:13.653942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:14.462905 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.462871    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-p5n2j"
Sep 10 14:21:14.553875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.553846    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"]
Sep 10 14:21:14.554073 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.554050    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" containerName="manager" containerID="cri-o://5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31" gracePeriod=2
Sep 10 14:21:14.565520 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.565493    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"]
Sep 10 14:21:14.565821 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.565781    2576 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"96a636d9-d2d3-4781-844c-351e4573e25c\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-gplbc\": pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.584826 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.584796    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"]
Sep 10 14:21:14.585174 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.585160    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" containerName="manager"
Sep 10 14:21:14.585214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.585177    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" containerName="manager"
Sep 10 14:21:14.585328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.585316    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" containerName="manager"
Sep 10 14:21:14.588148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.588130    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.590524 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.590498    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.603634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.603603    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"]
Sep 10 14:21:14.607156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.607133    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"]
Sep 10 14:21:14.607409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.607382    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" containerName="manager" containerID="cri-o://7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810" gracePeriod=2
Sep 10 14:21:14.609295 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.609271    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:14.622139 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.622110    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"]
Sep 10 14:21:14.627680 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.627647    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pbhbr\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:14.629835 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.629813    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.640288 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.640256    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-675np\" (UniqueName: \"kubernetes.io/projected/b324b208-e5f1-4639-9420-735561a87ae9-kube-api-access-675np\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jmcxd\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.640460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.640415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b324b208-e5f1-4639-9420-735561a87ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jmcxd\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.648921 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.648896    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"]
Sep 10 14:21:14.649245 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.649215    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" containerName="manager"
Sep 10 14:21:14.649245 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.649245    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" containerName="manager"
Sep 10 14:21:14.649386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.649305    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" containerName="manager"
Sep 10 14:21:14.652182 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.652164    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:14.654766 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.654734    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pbhbr\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:14.656930 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.656909    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.664617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.664597    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"]
Sep 10 14:21:14.741319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.741281    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgh8h\" (UniqueName: \"kubernetes.io/projected/1f8014f5-448d-4fcd-b1c8-d24e6be1293d-kube-api-access-bgh8h\") pod \"limitador-operator-controller-manager-74495c69f8-pmdgx\" (UID: \"1f8014f5-448d-4fcd-b1c8-d24e6be1293d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:14.741500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.741393    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b324b208-e5f1-4639-9420-735561a87ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jmcxd\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.741500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.741449    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-675np\" (UniqueName: \"kubernetes.io/projected/b324b208-e5f1-4639-9420-735561a87ae9-kube-api-access-675np\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jmcxd\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.742289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.742261    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b324b208-e5f1-4639-9420-735561a87ae9-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jmcxd\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.754029 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.753998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-675np\" (UniqueName: \"kubernetes.io/projected/b324b208-e5f1-4639-9420-735561a87ae9-kube-api-access-675np\") pod \"kuadrant-operator-controller-manager-76f8f686bf-jmcxd\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.835573 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.835535    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.837952 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.837918    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pbhbr\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:14.839987 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.839960    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.842592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.842573    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgh8h\" (UniqueName: \"kubernetes.io/projected/1f8014f5-448d-4fcd-b1c8-d24e6be1293d-kube-api-access-bgh8h\") pod \"limitador-operator-controller-manager-74495c69f8-pmdgx\" (UID: \"1f8014f5-448d-4fcd-b1c8-d24e6be1293d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:14.846761 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.846743    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:14.848946 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.848923    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:14.851017 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:14.850987    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pbhbr\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:14.855116 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.855086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgh8h\" (UniqueName: \"kubernetes.io/projected/1f8014f5-448d-4fcd-b1c8-d24e6be1293d-kube-api-access-bgh8h\") pod \"limitador-operator-controller-manager-74495c69f8-pmdgx\" (UID: \"1f8014f5-448d-4fcd-b1c8-d24e6be1293d\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:14.943627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.943595    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tmf7\" (UniqueName: \"kubernetes.io/projected/0ffdeb52-fe42-4770-98e3-d8f69c3820bc-kube-api-access-8tmf7\") pod \"0ffdeb52-fe42-4770-98e3-d8f69c3820bc\" (UID: \"0ffdeb52-fe42-4770-98e3-d8f69c3820bc\") "
Sep 10 14:21:14.943793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.943657    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96a636d9-d2d3-4781-844c-351e4573e25c-extensions-socket-volume\") pod \"96a636d9-d2d3-4781-844c-351e4573e25c\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") "
Sep 10 14:21:14.943858 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.943828    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ssnns\" (UniqueName: \"kubernetes.io/projected/96a636d9-d2d3-4781-844c-351e4573e25c-kube-api-access-ssnns\") pod \"96a636d9-d2d3-4781-844c-351e4573e25c\" (UID: \"96a636d9-d2d3-4781-844c-351e4573e25c\") "
Sep 10 14:21:14.944188 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.944158    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96a636d9-d2d3-4781-844c-351e4573e25c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "96a636d9-d2d3-4781-844c-351e4573e25c" (UID: "96a636d9-d2d3-4781-844c-351e4573e25c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:21:14.945958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.945935    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ffdeb52-fe42-4770-98e3-d8f69c3820bc-kube-api-access-8tmf7" (OuterVolumeSpecName: "kube-api-access-8tmf7") pod "0ffdeb52-fe42-4770-98e3-d8f69c3820bc" (UID: "0ffdeb52-fe42-4770-98e3-d8f69c3820bc"). InnerVolumeSpecName "kube-api-access-8tmf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:21:14.946048 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.946013    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96a636d9-d2d3-4781-844c-351e4573e25c-kube-api-access-ssnns" (OuterVolumeSpecName: "kube-api-access-ssnns") pod "96a636d9-d2d3-4781-844c-351e4573e25c" (UID: "96a636d9-d2d3-4781-844c-351e4573e25c"). InnerVolumeSpecName "kube-api-access-ssnns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:21:14.986262 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.986182    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:14.992306 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:14.992281    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:15.045136 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.045102    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tmf7\" (UniqueName: \"kubernetes.io/projected/0ffdeb52-fe42-4770-98e3-d8f69c3820bc-kube-api-access-8tmf7\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:21:15.045265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.045143    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/96a636d9-d2d3-4781-844c-351e4573e25c-extensions-socket-volume\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:21:15.045265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.045158    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ssnns\" (UniqueName: \"kubernetes.io/projected/96a636d9-d2d3-4781-844c-351e4573e25c-kube-api-access-ssnns\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:21:15.278570 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.278466    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"]
Sep 10 14:21:15.282963 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:21:15.282918    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f8014f5_448d_4fcd_b1c8_d24e6be1293d.slice/crio-82139ca5f49ee98652095a9a1108eb6aedc952c3da9fead3e3f00d1d7fa10ad1 WatchSource:0}: Error finding container 82139ca5f49ee98652095a9a1108eb6aedc952c3da9fead3e3f00d1d7fa10ad1: Status 404 returned error can't find the container with id 82139ca5f49ee98652095a9a1108eb6aedc952c3da9fead3e3f00d1d7fa10ad1
Sep 10 14:21:15.291563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.291548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"]
Sep 10 14:21:15.291925 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:21:15.291906    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb324b208_e5f1_4639_9420_735561a87ae9.slice/crio-0f051c3b5582cb42c0d0aff61523ecc2f1a0eabbddff97ba6adde66e50901130 WatchSource:0}: Error finding container 0f051c3b5582cb42c0d0aff61523ecc2f1a0eabbddff97ba6adde66e50901130: Status 404 returned error can't find the container with id 0f051c3b5582cb42c0d0aff61523ecc2f1a0eabbddff97ba6adde66e50901130
Sep 10 14:21:15.525810 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.525775    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx" event={"ID":"1f8014f5-448d-4fcd-b1c8-d24e6be1293d","Type":"ContainerStarted","Data":"1631b6ea933a6897a6a5581b4407f1b2bd0519727b0cf73ed26e601dfe4a49ba"}
Sep 10 14:21:15.526290 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.525829    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx" event={"ID":"1f8014f5-448d-4fcd-b1c8-d24e6be1293d","Type":"ContainerStarted","Data":"82139ca5f49ee98652095a9a1108eb6aedc952c3da9fead3e3f00d1d7fa10ad1"}
Sep 10 14:21:15.526290 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.525913    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:15.527563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.527536    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd" event={"ID":"b324b208-e5f1-4639-9420-735561a87ae9","Type":"ContainerStarted","Data":"cd4f1ad938b5dceb8c18a4eefddabd3e7cc72a8c388f09d517b7706493ea7444"}
Sep 10 14:21:15.527689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.527585    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd" event={"ID":"b324b208-e5f1-4639-9420-735561a87ae9","Type":"ContainerStarted","Data":"0f051c3b5582cb42c0d0aff61523ecc2f1a0eabbddff97ba6adde66e50901130"}
Sep 10 14:21:15.527689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.527613    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:15.528041 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:15.528006    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:15.528930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.528876    2576 generic.go:356] "Generic (PLEG): container finished" podID="96a636d9-d2d3-4781-844c-351e4573e25c" containerID="5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31" exitCode=0
Sep 10 14:21:15.528930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.528922    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:15.529105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.528975    2576 scope.go:117] "RemoveContainer" containerID="5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31"
Sep 10 14:21:15.530424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.530402    2576 generic.go:356] "Generic (PLEG): container finished" podID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" containerID="7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810" exitCode=0
Sep 10 14:21:15.530531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.530459    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:15.538413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.538278    2576 scope.go:117] "RemoveContainer" containerID="5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31"
Sep 10 14:21:15.538645 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:15.538619    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31\": container with ID starting with 5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31 not found: ID does not exist" containerID="5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31"
Sep 10 14:21:15.538738 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.538656    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31"} err="failed to get container status \"5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31\": rpc error: code = NotFound desc = could not find container \"5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31\": container with ID starting with 5491c50fc2f61f06d1c9e97e54018e5461c0da23753e27de98db3aebea0bae31 not found: ID does not exist"
Sep 10 14:21:15.538738 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.538675    2576 scope.go:117] "RemoveContainer" containerID="7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810"
Sep 10 14:21:15.546595 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.546575    2576 scope.go:117] "RemoveContainer" containerID="7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810"
Sep 10 14:21:15.546991 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:15.546964    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810\": container with ID starting with 7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810 not found: ID does not exist" containerID="7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810"
Sep 10 14:21:15.547099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.546997    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810"} err="failed to get container status \"7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810\": rpc error: code = NotFound desc = could not find container \"7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810\": container with ID starting with 7ae522504cd8975494333b57764062281e20436309766b3db4125e2418169810 not found: ID does not exist"
Sep 10 14:21:15.548412 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:15.548381    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pbhbr\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:15.548697 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.548654    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx" podStartSLOduration=1.548638318 podStartE2EDuration="1.548638318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:15.546401224 +0000 UTC m=+557.634858794" watchObservedRunningTime="2026-09-10 14:21:15.548638318 +0000 UTC m=+557.637095891"
Sep 10 14:21:15.551638 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:15.551607    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-gplbc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-gplbc"
Sep 10 14:21:15.571914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:15.571866    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd" podStartSLOduration=1.5718460269999999 podStartE2EDuration="1.571846027s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:15.570654966 +0000 UTC m=+557.659112539" watchObservedRunningTime="2026-09-10 14:21:15.571846027 +0000 UTC m=+557.660303598"
Sep 10 14:21:15.572874 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:21:15.572829    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pbhbr\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pbhbr"
Sep 10 14:21:16.427405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:16.427369    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ffdeb52-fe42-4770-98e3-d8f69c3820bc" path="/var/lib/kubelet/pods/0ffdeb52-fe42-4770-98e3-d8f69c3820bc/volumes"
Sep 10 14:21:16.427728 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:16.427715    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96a636d9-d2d3-4781-844c-351e4573e25c" path="/var/lib/kubelet/pods/96a636d9-d2d3-4781-844c-351e4573e25c/volumes"
Sep 10 14:21:19.989068 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:19.989032    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:20.091883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:20.091834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:22.971771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:22.971734    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:23.073526 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:23.073494    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:26.539818 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:26.539778    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pmdgx"
Sep 10 14:21:26.540201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:26.539856    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:27.867990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:27.867954    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:27.967305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:27.967263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:28.073884 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:28.073848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:30.909382 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:30.909347    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"]
Sep 10 14:21:30.909913 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:30.909582    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd" podUID="b324b208-e5f1-4639-9420-735561a87ae9" containerName="manager" containerID="cri-o://cd4f1ad938b5dceb8c18a4eefddabd3e7cc72a8c388f09d517b7706493ea7444" gracePeriod=10
Sep 10 14:21:31.103201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.103173    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"]
Sep 10 14:21:31.121995 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.121968    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"]
Sep 10 14:21:31.122120 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.122073    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.177954 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.177925    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h66q5\" (UniqueName: \"kubernetes.io/projected/75a5368a-83aa-413c-a287-d987cbd312f4-kube-api-access-h66q5\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q792g\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.178074 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.177968    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75a5368a-83aa-413c-a287-d987cbd312f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q792g\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.278563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.278526    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h66q5\" (UniqueName: \"kubernetes.io/projected/75a5368a-83aa-413c-a287-d987cbd312f4-kube-api-access-h66q5\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q792g\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.278737 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.278588    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75a5368a-83aa-413c-a287-d987cbd312f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q792g\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.278975 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.278957    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75a5368a-83aa-413c-a287-d987cbd312f4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q792g\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.290642 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.290605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h66q5\" (UniqueName: \"kubernetes.io/projected/75a5368a-83aa-413c-a287-d987cbd312f4-kube-api-access-h66q5\") pod \"kuadrant-operator-controller-manager-5769f99d9f-q792g\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.468212 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.468173    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:31.607567 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.607536    2576 generic.go:356] "Generic (PLEG): container finished" podID="b324b208-e5f1-4639-9420-735561a87ae9" containerID="cd4f1ad938b5dceb8c18a4eefddabd3e7cc72a8c388f09d517b7706493ea7444" exitCode=0
Sep 10 14:21:31.607698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.607679    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd" event={"ID":"b324b208-e5f1-4639-9420-735561a87ae9","Type":"ContainerDied","Data":"cd4f1ad938b5dceb8c18a4eefddabd3e7cc72a8c388f09d517b7706493ea7444"}
Sep 10 14:21:31.700070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.700044    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:31.762226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.762140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"]
Sep 10 14:21:31.766265 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:21:31.766214    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75a5368a_83aa_413c_a287_d987cbd312f4.slice/crio-ffd03729b7455b504d101f8c66166e24fd39ed663d62dd950259b61c291e36c3 WatchSource:0}: Error finding container ffd03729b7455b504d101f8c66166e24fd39ed663d62dd950259b61c291e36c3: Status 404 returned error can't find the container with id ffd03729b7455b504d101f8c66166e24fd39ed663d62dd950259b61c291e36c3
Sep 10 14:21:31.782596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.782575    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-675np\" (UniqueName: \"kubernetes.io/projected/b324b208-e5f1-4639-9420-735561a87ae9-kube-api-access-675np\") pod \"b324b208-e5f1-4639-9420-735561a87ae9\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") "
Sep 10 14:21:31.782695 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.782645    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b324b208-e5f1-4639-9420-735561a87ae9-extensions-socket-volume\") pod \"b324b208-e5f1-4639-9420-735561a87ae9\" (UID: \"b324b208-e5f1-4639-9420-735561a87ae9\") "
Sep 10 14:21:31.782994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.782970    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b324b208-e5f1-4639-9420-735561a87ae9-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b324b208-e5f1-4639-9420-735561a87ae9" (UID: "b324b208-e5f1-4639-9420-735561a87ae9"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:21:31.784747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.784727    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b324b208-e5f1-4639-9420-735561a87ae9-kube-api-access-675np" (OuterVolumeSpecName: "kube-api-access-675np") pod "b324b208-e5f1-4639-9420-735561a87ae9" (UID: "b324b208-e5f1-4639-9420-735561a87ae9"). InnerVolumeSpecName "kube-api-access-675np". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:21:31.883641 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.883606    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b324b208-e5f1-4639-9420-735561a87ae9-extensions-socket-volume\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:21:31.883744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:31.883645    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-675np\" (UniqueName: \"kubernetes.io/projected/b324b208-e5f1-4639-9420-735561a87ae9-kube-api-access-675np\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:21:32.614174 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.614142    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"
Sep 10 14:21:32.614643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.614143    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd" event={"ID":"b324b208-e5f1-4639-9420-735561a87ae9","Type":"ContainerDied","Data":"0f051c3b5582cb42c0d0aff61523ecc2f1a0eabbddff97ba6adde66e50901130"}
Sep 10 14:21:32.614643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.614310    2576 scope.go:117] "RemoveContainer" containerID="cd4f1ad938b5dceb8c18a4eefddabd3e7cc72a8c388f09d517b7706493ea7444"
Sep 10 14:21:32.616527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.616427    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g" event={"ID":"75a5368a-83aa-413c-a287-d987cbd312f4","Type":"ContainerStarted","Data":"220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848"}
Sep 10 14:21:32.616527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.616480    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g" event={"ID":"75a5368a-83aa-413c-a287-d987cbd312f4","Type":"ContainerStarted","Data":"ffd03729b7455b504d101f8c66166e24fd39ed663d62dd950259b61c291e36c3"}
Sep 10 14:21:32.616685 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.616615    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:32.640597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.640550    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g" podStartSLOduration=1.640534561 podStartE2EDuration="1.640534561s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:32.638585815 +0000 UTC m=+574.727043411" watchObservedRunningTime="2026-09-10 14:21:32.640534561 +0000 UTC m=+574.728992133"
Sep 10 14:21:32.654030 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.653994    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"]
Sep 10 14:21:32.658977 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:32.658950    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-jmcxd"]
Sep 10 14:21:34.331341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:34.331302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:34.393557 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:34.393526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:34.426934 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:34.426897    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b324b208-e5f1-4639-9420-735561a87ae9" path="/var/lib/kubelet/pods/b324b208-e5f1-4639-9420-735561a87ae9/volumes"
Sep 10 14:21:40.889014 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:40.888981    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:40.990453 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:40.990403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:43.624655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:43.624615    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:21:43.896968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:43.896880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:43.998545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:43.998510    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:47.377099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:47.377070    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:47.818836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:47.818795    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:50.719576 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:50.719533    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:50.827290 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:50.827261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:51.115531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:51.115439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:51.218000 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:51.217968    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:51.527115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:51.527077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:51.629858 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:51.629826    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:55.548608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:55.548573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:55.652533 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:55.652500    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:56.957202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:56.957163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:57.015963 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:57.015931    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:58.302202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:58.302175    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:21:58.304588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:21:58.304568    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:22:01.919289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:01.919248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:02.018119 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.018074    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:02.147965 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.147934    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:02.236657 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.236616    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:22:02.236973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.236959    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b324b208-e5f1-4639-9420-735561a87ae9" containerName="manager"
Sep 10 14:22:02.237023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.236977    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b324b208-e5f1-4639-9420-735561a87ae9" containerName="manager"
Sep 10 14:22:02.237061 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.237038    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b324b208-e5f1-4639-9420-735561a87ae9" containerName="manager"
Sep 10 14:22:02.239166 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.239141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.241457 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.241436    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 14:22:02.247617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.247591    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:22:02.262184 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.262154    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:22:02.335690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.335652    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88rpc\" (UniqueName: \"kubernetes.io/projected/f036dde9-ef00-417a-9b32-e0cb38ca8d04-kube-api-access-88rpc\") pod \"limitador-limitador-84994f7ffc-c57jv\" (UID: \"f036dde9-ef00-417a-9b32-e0cb38ca8d04\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.335897 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.335731    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f036dde9-ef00-417a-9b32-e0cb38ca8d04-config-file\") pod \"limitador-limitador-84994f7ffc-c57jv\" (UID: \"f036dde9-ef00-417a-9b32-e0cb38ca8d04\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.436271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.436219    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88rpc\" (UniqueName: \"kubernetes.io/projected/f036dde9-ef00-417a-9b32-e0cb38ca8d04-kube-api-access-88rpc\") pod \"limitador-limitador-84994f7ffc-c57jv\" (UID: \"f036dde9-ef00-417a-9b32-e0cb38ca8d04\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.436446 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.436301    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f036dde9-ef00-417a-9b32-e0cb38ca8d04-config-file\") pod \"limitador-limitador-84994f7ffc-c57jv\" (UID: \"f036dde9-ef00-417a-9b32-e0cb38ca8d04\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.436960 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.436940    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f036dde9-ef00-417a-9b32-e0cb38ca8d04-config-file\") pod \"limitador-limitador-84994f7ffc-c57jv\" (UID: \"f036dde9-ef00-417a-9b32-e0cb38ca8d04\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.444183 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.444151    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88rpc\" (UniqueName: \"kubernetes.io/projected/f036dde9-ef00-417a-9b32-e0cb38ca8d04-kube-api-access-88rpc\") pod \"limitador-limitador-84994f7ffc-c57jv\" (UID: \"f036dde9-ef00-417a-9b32-e0cb38ca8d04\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.549657 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.549566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:02.753592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.753560    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:02.820211 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.820141    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:22:02.823014 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:22:02.822980    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf036dde9_ef00_417a_9b32_e0cb38ca8d04.slice/crio-82ea68b59b2662daf654852221eea99f91de0f874b33f23c768f2a4cb7eaeb51 WatchSource:0}: Error finding container 82ea68b59b2662daf654852221eea99f91de0f874b33f23c768f2a4cb7eaeb51: Status 404 returned error can't find the container with id 82ea68b59b2662daf654852221eea99f91de0f874b33f23c768f2a4cb7eaeb51
Sep 10 14:22:02.840213 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:02.840191    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:03.514454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:03.514408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:03.614773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:03.614724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:03.766899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:03.766818    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv" event={"ID":"f036dde9-ef00-417a-9b32-e0cb38ca8d04","Type":"ContainerStarted","Data":"82ea68b59b2662daf654852221eea99f91de0f874b33f23c768f2a4cb7eaeb51"}
Sep 10 14:22:04.777951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:04.777909    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv" event={"ID":"f036dde9-ef00-417a-9b32-e0cb38ca8d04","Type":"ContainerStarted","Data":"e545612a28e491b0ea48be79351c41413c6cf8136e608f20b7068f1b202d0895"}
Sep 10 14:22:04.778367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:04.778051    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:04.794415 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:04.794369    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv" podStartSLOduration=1.4846654369999999 podStartE2EDuration="2.794353552s" totalImagesPullingTime="1.309688115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:22:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:22:04.793493545 +0000 UTC m=+606.881951126" watchObservedRunningTime="2026-09-10 14:22:04.794353552 +0000 UTC m=+606.882811124"
Sep 10 14:22:06.508044 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:06.508006    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:06.609166 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:06.609125    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:15.783772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:15.783742    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-c57jv"
Sep 10 14:22:15.801883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:15.801848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:16.198308 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:16.198274    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:20.511997 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:20.511919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:20.610512 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:20.610482    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:21.019433 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:21.019395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:33.509584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:33.509547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:34.408450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:34.408408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:40.910018 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:40.909948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:41.010222 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:41.010190    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:41.067327 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:41.067302    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:41.672891 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:41.672852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:44.964454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:44.964421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:45.067566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:22:45.067532    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:13.777392 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:13.777358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:13.874916 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:13.874882    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:17.483554 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:17.483515    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:19.506850 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:19.506814    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:23:20.294312 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:20.294276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:30.418803 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:30.418765    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:30.518298 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:30.518261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:33.234526 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:33.234496    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:33.612061 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:33.611971    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:23:41.133189 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:41.133153    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:41.810597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:41.810558    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:23:45.847817 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:45.847785    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:23:46.830731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:23:46.830697    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:24:03.162511 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:03.162478    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:13.607296 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.607253    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"]
Sep 10 14:24:13.609958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.609935    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.612425 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.612402    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 14:24:13.612590 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.612573    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 14:24:13.613439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.613414    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4c25c\""
Sep 10 14:24:13.613560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.613469    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 14:24:13.624635 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.624610    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"]
Sep 10 14:24:13.643120 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.643096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:13.692437 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.692407    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.692563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.692447    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gf54\" (UniqueName: \"kubernetes.io/projected/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-kube-api-access-9gf54\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.692563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.692467    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.692563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.692543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.692680 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.692611    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.692680 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.692639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.792943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.792904    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gf54\" (UniqueName: \"kubernetes.io/projected/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-kube-api-access-9gf54\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.792943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.792943    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.792980    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.793020    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.793047    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.793079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.793424    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793485 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.793463    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.793528 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.793505    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.795572 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.795543    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.795790 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.795771    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.801545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.801528    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gf54\" (UniqueName: \"kubernetes.io/projected/fe39854b-f12d-4880-9f7c-fba9b24a8eb5-kube-api-access-9gf54\") pod \"e2e-trlp-test-simulated-kserve-598466548d-td7j4\" (UID: \"fe39854b-f12d-4880-9f7c-fba9b24a8eb5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:13.921017 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:13.920979    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:14.195709 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:14.195602    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"]
Sep 10 14:24:14.198393 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:24:14.198368    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe39854b_f12d_4880_9f7c_fba9b24a8eb5.slice/crio-4c2f01a9346f476d2487833c070db546938195fc0c91bac670681c37b53232fa WatchSource:0}: Error finding container 4c2f01a9346f476d2487833c070db546938195fc0c91bac670681c37b53232fa: Status 404 returned error can't find the container with id 4c2f01a9346f476d2487833c070db546938195fc0c91bac670681c37b53232fa
Sep 10 14:24:14.199739 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:14.199719    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:24:14.369379 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:14.369347    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4" event={"ID":"fe39854b-f12d-4880-9f7c-fba9b24a8eb5","Type":"ContainerStarted","Data":"4c2f01a9346f476d2487833c070db546938195fc0c91bac670681c37b53232fa"}
Sep 10 14:24:15.004288 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:15.004226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:24:15.377199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:15.377161    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4" event={"ID":"fe39854b-f12d-4880-9f7c-fba9b24a8eb5","Type":"ContainerStarted","Data":"70829cfa76a67fd00d97febb106fc403151efada0636bc90abb1f270b623bca2"}
Sep 10 14:24:23.921867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:23.921831    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:23.922506 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:23.921900    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:23.934399 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:23.934372    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:23.955053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:23.955005    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4" podStartSLOduration=9.956863115 podStartE2EDuration="10.95499131s" totalImagesPullingTime="998.128195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:24:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:24:15.398946447 +0000 UTC m=+737.487404017" watchObservedRunningTime="2026-09-10 14:24:23.95499131 +0000 UTC m=+746.043448880"
Sep 10 14:24:24.428023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.427998    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-td7j4"
Sep 10 14:24:24.549047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.549013    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"]
Sep 10 14:24:24.553930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.553899    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.557860 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.557836    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 10 14:24:24.562664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.562643    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"]
Sep 10 14:24:24.570071 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.570048    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.570174 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.570087    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.570279 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.570255    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.570322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.570308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.570356 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.570336    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.570399 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.570384    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv69f\" (UniqueName: \"kubernetes.io/projected/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-kube-api-access-sv69f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.618362 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.618311    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:24.671478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671441    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv69f\" (UniqueName: \"kubernetes.io/projected/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-kube-api-access-sv69f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.671640 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671487    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.671640 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.671640 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.671640 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671598    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.671640 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.671998 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671973    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.672041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.671996    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.672041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.672007    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.674112 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.674079    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.674254 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.674221    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.684400 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.684350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv69f\" (UniqueName: \"kubernetes.io/projected/c60385e0-2eb1-4ccf-ba76-0e4e8750126a-kube-api-access-sv69f\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d\" (UID: \"c60385e0-2eb1-4ccf-ba76-0e4e8750126a\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:24.866376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:24.866340    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:25.163972 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:25.163946    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"]
Sep 10 14:24:25.166496 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:24:25.166470    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60385e0_2eb1_4ccf_ba76_0e4e8750126a.slice/crio-20b36a2f6cb1b9b8f123a2ccd28db37d75ddad9bd21ca007133ed5bb37c6aff7 WatchSource:0}: Error finding container 20b36a2f6cb1b9b8f123a2ccd28db37d75ddad9bd21ca007133ed5bb37c6aff7: Status 404 returned error can't find the container with id 20b36a2f6cb1b9b8f123a2ccd28db37d75ddad9bd21ca007133ed5bb37c6aff7
Sep 10 14:24:25.422425 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:25.422401    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d" event={"ID":"c60385e0-2eb1-4ccf-ba76-0e4e8750126a","Type":"ContainerStarted","Data":"20b36a2f6cb1b9b8f123a2ccd28db37d75ddad9bd21ca007133ed5bb37c6aff7"}
Sep 10 14:24:26.317403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:26.317364    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:24:26.429092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:26.429061    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d" event={"ID":"c60385e0-2eb1-4ccf-ba76-0e4e8750126a","Type":"ContainerStarted","Data":"ed93cb836f42a95dc1633fd0fdab326f0322c942a58cf4a00d6d7252b1aa5038"}
Sep 10 14:24:26.456405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:26.456351    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d" podStartSLOduration=2.215435738 podStartE2EDuration="2.456330185s" totalImagesPullingTime="240.894447ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:24:26.454029275 +0000 UTC m=+748.542486846" watchObservedRunningTime="2026-09-10 14:24:26.456330185 +0000 UTC m=+748.544787757"
Sep 10 14:24:34.080430 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.080394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.141851 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.141812    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.220112 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.220078    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.293866 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.293836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.741885 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.741852    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.842946 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.842912    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.866848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.866813    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:34.866995 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.866868    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:34.879063 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.879041    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:34.911176 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:34.911148    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:35.480666 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:35.480632    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d"
Sep 10 14:24:36.615496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:36.615460    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:24:40.908464 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.908431    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"]
Sep 10 14:24:40.912418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.912395    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:40.914942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.914922    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 10 14:24:40.922186 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.922165    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"]
Sep 10 14:24:40.938285 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.938260    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:40.991441 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.991405    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:40.991441 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.991445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e239519a-53dd-4eed-af49-3afa5463d150-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:40.991663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.991490    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:40.991663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.991515    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:40.991663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.991541    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:40.991663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:40.991608    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfwhg\" (UniqueName: \"kubernetes.io/projected/e239519a-53dd-4eed-af49-3afa5463d150-kube-api-access-wfwhg\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.092601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.092564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.092601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.092602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.092836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.092622    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.092836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.092650    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfwhg\" (UniqueName: \"kubernetes.io/projected/e239519a-53dd-4eed-af49-3afa5463d150-kube-api-access-wfwhg\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.092836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.092698    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.092836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.092713    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e239519a-53dd-4eed-af49-3afa5463d150-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.093051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.093028    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.093104 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.093032    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.093178 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.093158    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.095181 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.095161    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e239519a-53dd-4eed-af49-3afa5463d150-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.095394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.095377    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e239519a-53dd-4eed-af49-3afa5463d150-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.100653 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.100626    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfwhg\" (UniqueName: \"kubernetes.io/projected/e239519a-53dd-4eed-af49-3afa5463d150-kube-api-access-wfwhg\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz\" (UID: \"e239519a-53dd-4eed-af49-3afa5463d150\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.222643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.222600    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:41.502814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:41.502779    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"]
Sep 10 14:24:41.505208 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:24:41.505179    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode239519a_53dd_4eed_af49_3afa5463d150.slice/crio-3c27513f1788e3b65adfb7e5ca4a18e9f4667bfc45ed0bcd14382b5f28833e4b WatchSource:0}: Error finding container 3c27513f1788e3b65adfb7e5ca4a18e9f4667bfc45ed0bcd14382b5f28833e4b: Status 404 returned error can't find the container with id 3c27513f1788e3b65adfb7e5ca4a18e9f4667bfc45ed0bcd14382b5f28833e4b
Sep 10 14:24:42.504747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:42.504709    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz" event={"ID":"e239519a-53dd-4eed-af49-3afa5463d150","Type":"ContainerStarted","Data":"5bbb0af626e94bb081781959afb961568e523dd0d38d11c55340f56001e23892"}
Sep 10 14:24:42.505144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:42.504761    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz" event={"ID":"e239519a-53dd-4eed-af49-3afa5463d150","Type":"ContainerStarted","Data":"3c27513f1788e3b65adfb7e5ca4a18e9f4667bfc45ed0bcd14382b5f28833e4b"}
Sep 10 14:24:42.523695 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:42.523643    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz" podStartSLOduration=2.322368953 podStartE2EDuration="2.523629369s" totalImagesPullingTime="201.260416ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:24:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:24:42.521390308 +0000 UTC m=+764.609847878" watchObservedRunningTime="2026-09-10 14:24:42.523629369 +0000 UTC m=+764.612086940"
Sep 10 14:24:42.816750 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:42.816669    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:24:45.333842 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:45.333788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:45.435505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:45.435469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:45.504618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:45.504585    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:46.729261 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:46.729185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:51.222841 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:51.222798    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:51.222841 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:51.222843    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:51.235332 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:51.235307    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:24:51.558289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:24:51.558183    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz"
Sep 10 14:25:31.161548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:25:31.161413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:36.858189 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:25:36.858108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:57.750726 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:25:57.750684    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:57.845540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:25:57.845493    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:58.861650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:25:58.861609    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:58.962007 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:25:58.961973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:02.951588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:02.951550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:21.700715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:21.700680    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:21.804838 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:21.804800    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:22.050024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:22.049948    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:22.297027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:22.296988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:23.133863 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.133833    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:23.233833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.233799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:23.622637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.622597    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:23.774271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.774222    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7608-0d581f"]
Sep 10 14:26:23.777432 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.777416    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:23.780667 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.780645    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2hkkb\""
Sep 10 14:26:23.783320 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.783297    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7608-0d581f"]
Sep 10 14:26:23.887451 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.887361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fz7w7\" (UniqueName: \"kubernetes.io/projected/1d418fbc-342a-4dbf-9b77-176e7491aeaf-kube-api-access-fz7w7\") pod \"test-curl-7608-0d581f\" (UID: \"1d418fbc-342a-4dbf-9b77-176e7491aeaf\") " pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:23.988592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.988556    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fz7w7\" (UniqueName: \"kubernetes.io/projected/1d418fbc-342a-4dbf-9b77-176e7491aeaf-kube-api-access-fz7w7\") pod \"test-curl-7608-0d581f\" (UID: \"1d418fbc-342a-4dbf-9b77-176e7491aeaf\") " pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:23.997264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:23.997212    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fz7w7\" (UniqueName: \"kubernetes.io/projected/1d418fbc-342a-4dbf-9b77-176e7491aeaf-kube-api-access-fz7w7\") pod \"test-curl-7608-0d581f\" (UID: \"1d418fbc-342a-4dbf-9b77-176e7491aeaf\") " pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:24.087663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:24.087633    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:24.357957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:24.357927    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7608-0d581f"]
Sep 10 14:26:24.362491 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:26:24.362463    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d418fbc_342a_4dbf_9b77_176e7491aeaf.slice/crio-6ae530394406999d6dac35e85d63aecb841ca8429f766f068e8a12d3edc26550 WatchSource:0}: Error finding container 6ae530394406999d6dac35e85d63aecb841ca8429f766f068e8a12d3edc26550: Status 404 returned error can't find the container with id 6ae530394406999d6dac35e85d63aecb841ca8429f766f068e8a12d3edc26550
Sep 10 14:26:24.974906 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:24.974872    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7608-0d581f" event={"ID":"1d418fbc-342a-4dbf-9b77-176e7491aeaf","Type":"ContainerStarted","Data":"6ae530394406999d6dac35e85d63aecb841ca8429f766f068e8a12d3edc26550"}
Sep 10 14:26:25.113307 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:25.113268    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:26:25.886779 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:25.886748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:25.984539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:25.984489    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:27.047971 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:27.047935    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:27.163502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:27.163468    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:27.381649 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:27.381557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:27.618891 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:27.618855    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:27.719831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:27.719799    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:29.568736 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:29.568702    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:29.669653 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:29.669616    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:29.784418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:29.784389    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:30.081763 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:30.081729    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:30.673565 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:30.673529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:33.080683 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:33.080641    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:33.179740 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:33.179691    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.238494 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.238457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.492725 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.492659    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"]
Sep 10 14:26:34.496014 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.495997    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.498539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.498520    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-a-9a074211-kserve-self-signed-certs\""
Sep 10 14:26:34.509269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.507138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"]
Sep 10 14:26:34.587876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.587845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-model-cache\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.588057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.587886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd72q\" (UniqueName: \"kubernetes.io/projected/4da36acb-99a6-4866-9072-8f502a41215c-kube-api-access-xd72q\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.588057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.587909    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-tmp-dir\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.588057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.587941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-home\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.588057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.588010    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-dshm\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.588214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.588081    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4da36acb-99a6-4866-9072-8f502a41215c-tls-certs\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.655289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.655220    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.689508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689478    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4da36acb-99a6-4866-9072-8f502a41215c-tls-certs\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.689671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689536    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-model-cache\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.689671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689555    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd72q\" (UniqueName: \"kubernetes.io/projected/4da36acb-99a6-4866-9072-8f502a41215c-kube-api-access-xd72q\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.689671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689571    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-tmp-dir\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.689671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689594    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-home\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.689671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689613    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-dshm\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.689951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.689932    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-model-cache\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.690388 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.690360    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-tmp-dir\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.690388 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.690375    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-home\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.692227 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.692207    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-dshm\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.692528 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.692502    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4da36acb-99a6-4866-9072-8f502a41215c-tls-certs\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.697907 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.697889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd72q\" (UniqueName: \"kubernetes.io/projected/4da36acb-99a6-4866-9072-8f502a41215c-kube-api-access-xd72q\") pod \"e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") " pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.753656 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.753563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.809323 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.809289    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:34.852270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.852163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.998821 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:34.998788    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:35.029614 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.029535    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7608-0d581f" event={"ID":"1d418fbc-342a-4dbf-9b77-176e7491aeaf","Type":"ContainerStarted","Data":"9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd"}
Sep 10 14:26:35.043898 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.043856    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7608-0d581f" podStartSLOduration=2.003539179 podStartE2EDuration="12.043837072s" totalImagesPullingTime="10.040297893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:35.043301665 +0000 UTC m=+877.131759234" watchObservedRunningTime="2026-09-10 14:26:35.043837072 +0000 UTC m=+877.132294643"
Sep 10 14:26:35.087553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.087530    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"]
Sep 10 14:26:35.090024 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:26:35.089997    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4da36acb_99a6_4866_9072_8f502a41215c.slice/crio-9fc6dd617956c7baea3a58c2797a33df20f800c50c0c5021e5cdeb4d3a228792 WatchSource:0}: Error finding container 9fc6dd617956c7baea3a58c2797a33df20f800c50c0c5021e5cdeb4d3a228792: Status 404 returned error can't find the container with id 9fc6dd617956c7baea3a58c2797a33df20f800c50c0c5021e5cdeb4d3a228792
Sep 10 14:26:35.710144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.710116    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7608-0d581f"]
Sep 10 14:26:35.714218 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.714190    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7608-0d581f"]
Sep 10 14:26:35.842757 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.842677    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7608-643811"]
Sep 10 14:26:35.843041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.843028    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" containerName="test-curl-7608-0d581f"
Sep 10 14:26:35.843083 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.843043    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" containerName="test-curl-7608-0d581f"
Sep 10 14:26:35.843117 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.843094    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" containerName="test-curl-7608-0d581f"
Sep 10 14:26:35.846118 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.846098    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:35.853151 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.852942    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7608-643811"]
Sep 10 14:26:35.899710 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:35.899679    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kh8s8\" (UniqueName: \"kubernetes.io/projected/ad4f48a9-fef2-4b75-854a-545bcb240c2a-kube-api-access-kh8s8\") pod \"test-curl-7608-643811\" (UID: \"ad4f48a9-fef2-4b75-854a-545bcb240c2a\") " pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:36.001031 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.000991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kh8s8\" (UniqueName: \"kubernetes.io/projected/ad4f48a9-fef2-4b75-854a-545bcb240c2a-kube-api-access-kh8s8\") pod \"test-curl-7608-643811\" (UID: \"ad4f48a9-fef2-4b75-854a-545bcb240c2a\") " pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:36.010881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.010854    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kh8s8\" (UniqueName: \"kubernetes.io/projected/ad4f48a9-fef2-4b75-854a-545bcb240c2a-kube-api-access-kh8s8\") pod \"test-curl-7608-643811\" (UID: \"ad4f48a9-fef2-4b75-854a-545bcb240c2a\") " pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:36.035298 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.035267    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z" event={"ID":"4da36acb-99a6-4866-9072-8f502a41215c","Type":"ContainerStarted","Data":"c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d"}
Sep 10 14:26:36.035418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.035308    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z" event={"ID":"4da36acb-99a6-4866-9072-8f502a41215c","Type":"ContainerStarted","Data":"9fc6dd617956c7baea3a58c2797a33df20f800c50c0c5021e5cdeb4d3a228792"}
Sep 10 14:26:36.035576 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:26:36.035556    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7608-0d581f" container="test-curl-7608-0d581f"
Sep 10 14:26:36.053875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.053738    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z" podStartSLOduration=2.05371854 podStartE2EDuration="2.05371854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:36.053136491 +0000 UTC m=+878.141594065" watchObservedRunningTime="2026-09-10 14:26:36.05371854 +0000 UTC m=+878.142176112"
Sep 10 14:26:36.158973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.158881    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:36.429422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:36.429400    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7608-643811"]
Sep 10 14:26:36.431532 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:26:36.431509    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad4f48a9_fef2_4b75_854a_545bcb240c2a.slice/crio-cb429145b50fb8c48c6b10f3e6b372c0dc2a092970e1d948f19d68008a7b302c WatchSource:0}: Error finding container cb429145b50fb8c48c6b10f3e6b372c0dc2a092970e1d948f19d68008a7b302c: Status 404 returned error can't find the container with id cb429145b50fb8c48c6b10f3e6b372c0dc2a092970e1d948f19d68008a7b302c
Sep 10 14:26:37.042194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:37.042160    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7608-643811" event={"ID":"ad4f48a9-fef2-4b75-854a-545bcb240c2a","Type":"ContainerStarted","Data":"99c5a3ed94a29151c706f9d5702dcd28addcefb657bfcae1e043c2326374bd06"}
Sep 10 14:26:37.042194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:37.042198    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7608-643811" event={"ID":"ad4f48a9-fef2-4b75-854a-545bcb240c2a","Type":"ContainerStarted","Data":"cb429145b50fb8c48c6b10f3e6b372c0dc2a092970e1d948f19d68008a7b302c"}
Sep 10 14:26:37.042735 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:37.042382    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7608-0d581f" podUID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" containerName="test-curl-7608-0d581f" containerID="cri-o://9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd" gracePeriod=2
Sep 10 14:26:37.057171 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:37.057121    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7608-643811" podStartSLOduration=2.057103487 podStartE2EDuration="2.057103487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:37.055522469 +0000 UTC m=+879.143980039" watchObservedRunningTime="2026-09-10 14:26:37.057103487 +0000 UTC m=+879.145561060"
Sep 10 14:26:37.676009 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:37.675969    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7608-643811"]
Sep 10 14:26:37.678082 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:37.678048    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7608-643811"]
Sep 10 14:26:38.051285 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:26:38.048727    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7608-643811" container="test-curl-7608-643811"
Sep 10 14:26:38.206767 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.206735    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:38.497627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.497594    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"]
Sep 10 14:26:38.497965 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.497949    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad4f48a9-fef2-4b75-854a-545bcb240c2a" containerName="test-curl-7608-643811"
Sep 10 14:26:38.498032 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.497968    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad4f48a9-fef2-4b75-854a-545bcb240c2a" containerName="test-curl-7608-643811"
Sep 10 14:26:38.498086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.498058    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad4f48a9-fef2-4b75-854a-545bcb240c2a" containerName="test-curl-7608-643811"
Sep 10 14:26:38.510699 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.510673    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.514265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.514222    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:26:38.516280 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.515965    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"]
Sep 10 14:26:38.516481 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.516458    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-44e3af-openshift-default-dockercfg-562wx\""
Sep 10 14:26:38.538763 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.538739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:38.621291 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621259    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-credential-socket\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621291 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621295    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvt65\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-kube-api-access-bvt65\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621516 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621321    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istiod-ca-cert\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621516 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621449    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-podinfo\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621516 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-socket\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621552    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-token\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-data\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621607    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-envoy\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.621687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.621625    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-certs\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722035 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722000    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-data\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722044    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-envoy\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722065    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-certs\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722085    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-credential-socket\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722109    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvt65\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-kube-api-access-bvt65\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722143    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istiod-ca-cert\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722192    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-podinfo\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722500    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-data\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722594    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-certs\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722717 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722675    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-credential-socket\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722717 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722687    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-socket\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-token\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.722986 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.722959    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-socket\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.723104 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.723008    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istiod-ca-cert\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.724651 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.724627    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-envoy\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.725198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.725174    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-podinfo\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.732592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.732574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-token\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.732834 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.732808    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvt65\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-kube-api-access-bvt65\") pod \"e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") " pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:38.825539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:38.825452    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:39.054531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.054128    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7608-643811" podUID="ad4f48a9-fef2-4b75-854a-545bcb240c2a" containerName="test-curl-7608-643811" containerID="cri-o://99c5a3ed94a29151c706f9d5702dcd28addcefb657bfcae1e043c2326374bd06" gracePeriod=2
Sep 10 14:26:39.112620 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.112570    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"]
Sep 10 14:26:39.115604 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:26:39.115572    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc04d23d2_40a6_4666_bd5e_3dc89cf52ad2.slice/crio-94c350575f4ae4e8ffdc8e98a4779292d4cf087fa095d330c8df0696264cb272 WatchSource:0}: Error finding container 94c350575f4ae4e8ffdc8e98a4779292d4cf087fa095d330c8df0696264cb272: Status 404 returned error can't find the container with id 94c350575f4ae4e8ffdc8e98a4779292d4cf087fa095d330c8df0696264cb272
Sep 10 14:26:39.207630 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.207606    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:39.237972 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.237947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:39.326249 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.326197    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:26:39.328897 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.328873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fz7w7\" (UniqueName: \"kubernetes.io/projected/1d418fbc-342a-4dbf-9b77-176e7491aeaf-kube-api-access-fz7w7\") pod \"1d418fbc-342a-4dbf-9b77-176e7491aeaf\" (UID: \"1d418fbc-342a-4dbf-9b77-176e7491aeaf\") "
Sep 10 14:26:39.331177 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.331131    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d418fbc-342a-4dbf-9b77-176e7491aeaf-kube-api-access-fz7w7" (OuterVolumeSpecName: "kube-api-access-fz7w7") pod "1d418fbc-342a-4dbf-9b77-176e7491aeaf" (UID: "1d418fbc-342a-4dbf-9b77-176e7491aeaf"). InnerVolumeSpecName "kube-api-access-fz7w7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:26:39.430352 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:39.430314    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fz7w7\" (UniqueName: \"kubernetes.io/projected/1d418fbc-342a-4dbf-9b77-176e7491aeaf-kube-api-access-fz7w7\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:26:40.060850 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.060759    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" event={"ID":"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2","Type":"ContainerStarted","Data":"94c350575f4ae4e8ffdc8e98a4779292d4cf087fa095d330c8df0696264cb272"}
Sep 10 14:26:40.062050 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.062017    2576 generic.go:356] "Generic (PLEG): container finished" podID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" containerID="9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd" exitCode=137
Sep 10 14:26:40.062185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.062095    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:40.062185 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.062117    2576 scope.go:117] "RemoveContainer" containerID="9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd"
Sep 10 14:26:40.071658 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.071634    2576 scope.go:117] "RemoveContainer" containerID="9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd"
Sep 10 14:26:40.071938 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:26:40.071920    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd\": container with ID starting with 9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd not found: ID does not exist" containerID="9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd"
Sep 10 14:26:40.072001 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.071946    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd"} err="failed to get container status \"9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd\": rpc error: code = NotFound desc = could not find container \"9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd\": container with ID starting with 9633786d1deaeee31218bdea51726085a8990e156bc2848c804e270522aed9cd not found: ID does not exist"
Sep 10 14:26:40.073134 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:26:40.073110    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7608-0d581f\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" pod="opendatahub/test-curl-7608-0d581f"
Sep 10 14:26:40.429369 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.429328    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d418fbc-342a-4dbf-9b77-176e7491aeaf" path="/var/lib/kubelet/pods/1d418fbc-342a-4dbf-9b77-176e7491aeaf/volumes"
Sep 10 14:26:40.462611 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.462578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:40.617825 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.617794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:40.738152 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.738063    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:40.834655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:40.834617    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:42.077214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.077177    2576 generic.go:356] "Generic (PLEG): container finished" podID="ad4f48a9-fef2-4b75-854a-545bcb240c2a" containerID="99c5a3ed94a29151c706f9d5702dcd28addcefb657bfcae1e043c2326374bd06" exitCode=137
Sep 10 14:26:42.233414 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.233382    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:42.268749 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.268705    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:26:42.268838 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.268824    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:26:42.268885 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.268874    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:26:42.358166 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.358075    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kh8s8\" (UniqueName: \"kubernetes.io/projected/ad4f48a9-fef2-4b75-854a-545bcb240c2a-kube-api-access-kh8s8\") pod \"ad4f48a9-fef2-4b75-854a-545bcb240c2a\" (UID: \"ad4f48a9-fef2-4b75-854a-545bcb240c2a\") "
Sep 10 14:26:42.360822 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.360785    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad4f48a9-fef2-4b75-854a-545bcb240c2a-kube-api-access-kh8s8" (OuterVolumeSpecName: "kube-api-access-kh8s8") pod "ad4f48a9-fef2-4b75-854a-545bcb240c2a" (UID: "ad4f48a9-fef2-4b75-854a-545bcb240c2a"). InnerVolumeSpecName "kube-api-access-kh8s8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:26:42.429006 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.428970    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad4f48a9-fef2-4b75-854a-545bcb240c2a" path="/var/lib/kubelet/pods/ad4f48a9-fef2-4b75-854a-545bcb240c2a/volumes"
Sep 10 14:26:42.459612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:42.459578    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kh8s8\" (UniqueName: \"kubernetes.io/projected/ad4f48a9-fef2-4b75-854a-545bcb240c2a-kube-api-access-kh8s8\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:26:43.085177 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.085147    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7608-643811"
Sep 10 14:26:43.085618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.085199    2576 scope.go:117] "RemoveContainer" containerID="99c5a3ed94a29151c706f9d5702dcd28addcefb657bfcae1e043c2326374bd06"
Sep 10 14:26:43.086994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.086960    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" event={"ID":"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2","Type":"ContainerStarted","Data":"f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"}
Sep 10 14:26:43.116689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.116641    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podStartSLOduration=1.966367371 podStartE2EDuration="5.116623812s" totalImagesPullingTime="3.150256441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:43.113377495 +0000 UTC m=+885.201835068" watchObservedRunningTime="2026-09-10 14:26:43.116623812 +0000 UTC m=+885.205081457"
Sep 10 14:26:43.526971 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.526938    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:43.826294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.826182    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:43.830861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:43.830836    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:44.094305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:44.094211    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:44.095116 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:44.095105    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:26:44.810303 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:44.810270    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:44.810303 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:44.810308    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:44.822984 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:44.822959    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:45.057606 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:45.057573    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:45.110946 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:45.110862    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:26:45.141217 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:45.141187    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:45.957137 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:45.957106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:46.202109 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:46.202072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:46.303702 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:46.303589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:46.404101 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:46.404049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:46.671053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:46.671029    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:46.775863 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:46.775834    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:48.098249 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:48.098199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:50.716521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:50.716435    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:51.031593 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:51.031506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:51.325020 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:51.324942    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:51.425984 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:51.425947    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:54.804942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:54.804897    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:54.899581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:54.899542    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:55.313852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:55.313821    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:55.508801 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:55.508770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:55.604533 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:55.604452    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:55.705810 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:55.705772    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:56.235898 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.235862    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"]
Sep 10 14:26:56.242527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.242493    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.246032 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.246008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:26:56.246032 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.246022    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:26:56.246264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.246103    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-8f0c5f-serving-cert\""
Sep 10 14:26:56.246264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.246008    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:26:56.246357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.246333    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:26:56.252007 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.251982    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"]
Sep 10 14:26:56.282054 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.282014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:56.300895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.300855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11da1529-b807-4403-a07d-7121e47d8959-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.301058 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.300938    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.301058 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.300966    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fx8sc\" (UniqueName: \"kubernetes.io/projected/11da1529-b807-4403-a07d-7121e47d8959-kube-api-access-fx8sc\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.301058 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.301027    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-metrics-certs\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.383502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.383473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:56.401871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.401825    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.402048 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.401878    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fx8sc\" (UniqueName: \"kubernetes.io/projected/11da1529-b807-4403-a07d-7121e47d8959-kube-api-access-fx8sc\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.402048 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.401907    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-metrics-certs\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.402048 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:26:56.401985    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-serving-cert: secret "maas-api-e2e-inf-b-8f0c5f-serving-cert" not found
Sep 10 14:26:56.402256 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:26:56.402082    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls podName:11da1529-b807-4403-a07d-7121e47d8959 nodeName:}" failed. No retries permitted until 2026-09-10 14:26:56.902059292 +0000 UTC m=+898.990516842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls") pod "maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" (UID: "11da1529-b807-4403-a07d-7121e47d8959") : secret "maas-api-e2e-inf-b-8f0c5f-serving-cert" not found
Sep 10 14:26:56.402256 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.401994    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11da1529-b807-4403-a07d-7121e47d8959-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.402731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.402698    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11da1529-b807-4403-a07d-7121e47d8959-openshift-service-ca\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.404999 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.404944    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-metrics-certs\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.411749 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.411718    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fx8sc\" (UniqueName: \"kubernetes.io/projected/11da1529-b807-4403-a07d-7121e47d8959-kube-api-access-fx8sc\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.532559 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.532451    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:56.747387 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.747356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:56.847612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.847512    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:56.906701 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.906668    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:56.909203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:56.909170    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls\") pod \"maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:57.159954 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:57.159861    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:26:57.255386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:57.255210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:57.436643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:57.436611    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"]
Sep 10 14:26:57.440904 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:26:57.440843    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11da1529_b807_4403_a07d_7121e47d8959.slice/crio-b28290173bee89b5e579e826785010b8608f39ec357a0fa0effe5d431cd716ec WatchSource:0}: Error finding container b28290173bee89b5e579e826785010b8608f39ec357a0fa0effe5d431cd716ec: Status 404 returned error can't find the container with id b28290173bee89b5e579e826785010b8608f39ec357a0fa0effe5d431cd716ec
Sep 10 14:26:58.013297 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:58.013263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:58.186476 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:58.180384    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" event={"ID":"11da1529-b807-4403-a07d-7121e47d8959","Type":"ContainerStarted","Data":"b28290173bee89b5e579e826785010b8608f39ec357a0fa0effe5d431cd716ec"}
Sep 10 14:26:58.350812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:58.350732    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:26:58.354935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:58.354724    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:26:58.752407 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:58.752356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:59.096482 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.096395    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:26:59.096680 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.096476    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:26:59.187897 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.187865    2576 generic.go:356] "Generic (PLEG): container finished" podID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerID="f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d" exitCode=0
Sep 10 14:26:59.188069 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.187939    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" event={"ID":"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2","Type":"ContainerDied","Data":"f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"}
Sep 10 14:26:59.188454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.188426    2576 scope.go:117] "RemoveContainer" containerID="f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"
Sep 10 14:26:59.332679 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.332623    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:26:59.332833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.332724    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:26:59.332833 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:26:59.332760    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:00.195202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.195164    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" event={"ID":"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2","Type":"ContainerStarted","Data":"c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331"}
Sep 10 14:27:00.196764 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.196741    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" event={"ID":"11da1529-b807-4403-a07d-7121e47d8959","Type":"ContainerStarted","Data":"788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21"}
Sep 10 14:27:00.196899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.196807    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:27:00.233559 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.233510    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" podStartSLOduration=1.90583894 podStartE2EDuration="4.233494506s" totalImagesPullingTime="2.327655566s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:27:00.23241077 +0000 UTC m=+902.320868343" watchObservedRunningTime="2026-09-10 14:27:00.233494506 +0000 UTC m=+902.321952077"
Sep 10 14:27:00.826497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.826457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:27:00.826731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.826711    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:27:00.826777 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:00.826755    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:27:01.416480 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:01.416445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:27:01.733069 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:01.733037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:01.826817 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:01.826783    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:27:01.826990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:01.826868    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:27:01.885677 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:01.885647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:01.983450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:01.983361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:02.099222 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:02.099185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:02.826492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:02.826456    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:27:02.826903 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:02.826513    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:27:03.825983 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:03.825946    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:27:03.826205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:03.826016    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:27:04.826657 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:04.826621    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:27:04.827136 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:04.826700    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:27:05.826681 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:05.826643    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Sep 10 14:27:05.827091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:05.826706    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Sep 10 14:27:06.207561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:06.207531    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:27:06.830537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:06.830507    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:27:06.830899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:06.830775    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:27:06.831560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:06.831543    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:27:07.171149 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:07.171112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:08.052450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:08.052406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:08.154915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:08.154880    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:09.992011 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:09.991969    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:14.381386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:14.381351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:14.899127 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:14.899084    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:14.977287 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:14.977224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:15.053284 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:15.053253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:15.117558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:15.117516    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:15.220871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:15.220836    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:15.687304 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:15.687273    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:15.839624 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:15.839565    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:15.975497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:15.975397    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:16.177540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.177505    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"]
Sep 10 14:27:16.184466 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.184444    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.187870 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.187843    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-59ea8af5-gw-openshift-default-dockercfg-rhggn\""
Sep 10 14:27:16.195197 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.195169    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"]
Sep 10 14:27:16.208760 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.208733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:16.289917 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.289829    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-socket\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.289917 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.289884    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-podinfo\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.289922    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istiod-ca-cert\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.289941    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4l54\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-kube-api-access-j4l54\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.289993    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-envoy\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.290011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-data\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.290073    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-certs\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290115 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.290091    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-token\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.290408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.290201    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-credential-socket\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.318823 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.318794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:16.391353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391320    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-envoy\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-data\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391418    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-certs\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391443    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-token\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391502    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-credential-socket\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391545    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-socket\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391689    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-podinfo\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391756    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istiod-ca-cert\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-data\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391870    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-certs\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.391909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4l54\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-kube-api-access-j4l54\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.392046 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.391922    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-credential-socket\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.392128 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.392102    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-socket\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.392709 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.392686    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istiod-ca-cert\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.393906 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.393890    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-envoy\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.394330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.394313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-podinfo\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.400083 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.400055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-token\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.400369 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.400350    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4l54\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-kube-api-access-j4l54\") pod \"e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") " pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.498493 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.498454    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:16.773871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.773781    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"]
Sep 10 14:27:16.776756 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:27:16.776714    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116e3ad9_7af9_4c92_bb0d_ef1b8d35e939.slice/crio-854d75a560c155f0349593afc2df05289aa8ae0346a7406f48d1a06df2695f47 WatchSource:0}: Error finding container 854d75a560c155f0349593afc2df05289aa8ae0346a7406f48d1a06df2695f47: Status 404 returned error can't find the container with id 854d75a560c155f0349593afc2df05289aa8ae0346a7406f48d1a06df2695f47
Sep 10 14:27:16.779111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.779074    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:16.779220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.779156    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:16.779220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:16.779198    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:17.253216 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:17.253183    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:17.279879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:17.279843    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" event={"ID":"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939","Type":"ContainerStarted","Data":"daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"}
Sep 10 14:27:17.280175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:17.279897    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" event={"ID":"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939","Type":"ContainerStarted","Data":"854d75a560c155f0349593afc2df05289aa8ae0346a7406f48d1a06df2695f47"}
Sep 10 14:27:17.304950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:17.304892    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podStartSLOduration=1.304872106 podStartE2EDuration="1.304872106s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:27:17.303551136 +0000 UTC m=+919.392008709" watchObservedRunningTime="2026-09-10 14:27:17.304872106 +0000 UTC m=+919.393329677"
Sep 10 14:27:17.498643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:17.498613    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:17.503638 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:17.503563    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:18.284944 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:18.284917    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:18.285951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:18.285932    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:18.406111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:18.406075    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:27:18.866428 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:18.866396    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:18.957149 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:18.957111    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:19.026583 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:19.026543    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:19.132889 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:19.132814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:19.512609 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:19.512580    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:19.582137 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:19.582101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:19.736398 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:19.736361    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:37.941214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:37.941174    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:27:38.399671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:38.399573    2576 generic.go:356] "Generic (PLEG): container finished" podID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerID="daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb" exitCode=0
Sep 10 14:27:38.399671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:38.399655    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" event={"ID":"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939","Type":"ContainerDied","Data":"daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"}
Sep 10 14:27:38.400036 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:38.400021    2576 scope.go:117] "RemoveContainer" containerID="daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"
Sep 10 14:27:38.401063 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:38.401035    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:38.401124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:38.401105    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:38.401159 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:38.401140    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:27:39.407302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:39.407265    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" event={"ID":"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939","Type":"ContainerStarted","Data":"4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf"}
Sep 10 14:27:39.498925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:39.498893    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:39.499136 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:39.499113    2576 patch_prober.go:28] interesting pod/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 14:27:39.499201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:39.499164    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 14:27:40.499314 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:40.499271    2576 patch_prober.go:28] interesting pod/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 14:27:40.499702 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:40.499343    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 14:27:41.499535 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:41.499489    2576 patch_prober.go:28] interesting pod/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 14:27:41.499915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:41.499554    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 14:27:42.499345 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:42.499295    2576 patch_prober.go:28] interesting pod/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 14:27:42.499543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:42.499367    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 14:27:43.499502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:43.499464    2576 patch_prober.go:28] interesting pod/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Sep 10 14:27:43.499908 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:43.499531    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Sep 10 14:27:44.656819 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:44.656783    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:44.657297 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:44.657131    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:44.658041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:44.658018    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:50.561967 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.561902    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:50.661707 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.661664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:50.743282 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.743213    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:50.809927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.809895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:50.872619 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.872520    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"]
Sep 10 14:27:50.872793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.872763    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy" containerID="cri-o://4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf" gracePeriod=30
Sep 10 14:27:50.901925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:50.901899    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:51.333704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:51.333668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:51.674595 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:51.674562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:51.813891 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:51.813859    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:53.743291 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:53.743226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:53.845058 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:53.845021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:54.003409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.003332    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"]
Sep 10 14:27:54.005678 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.005661    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.008214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.008187    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-conflict-b-9a074211-kserve-self-signed-certs\""
Sep 10 14:27:54.014629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.014601    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"]
Sep 10 14:27:54.035487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.035458    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-model-cache\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.035637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.035494    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35779236-c2e0-4a73-b006-12028baad464-tls-certs\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.035637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.035549    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-dshm\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.035637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.035594    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-tmp-dir\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.035637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.035610    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w878c\" (UniqueName: \"kubernetes.io/projected/35779236-c2e0-4a73-b006-12028baad464-kube-api-access-w878c\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.035797 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.035664    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-home\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136443 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136406    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-home\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136463    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-model-cache\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136579    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35779236-c2e0-4a73-b006-12028baad464-tls-certs\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136756 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136697    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-dshm\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136756 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-model-cache\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-tmp-dir\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136792    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w878c\" (UniqueName: \"kubernetes.io/projected/35779236-c2e0-4a73-b006-12028baad464-kube-api-access-w878c\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.136859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.136806    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-home\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.137106 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.137086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-tmp-dir\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.139067 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.139049    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-dshm\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.139249 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.139214    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35779236-c2e0-4a73-b006-12028baad464-tls-certs\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.144146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.144128    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w878c\" (UniqueName: \"kubernetes.io/projected/35779236-c2e0-4a73-b006-12028baad464-kube-api-access-w878c\") pod \"e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") " pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.202377 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.202349    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:54.315604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.315508    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:27:54.592578 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.592543    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"]
Sep 10 14:27:54.594509 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:27:54.594486    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35779236_c2e0_4a73_b006_12028baad464.slice/crio-0407730e2b50638ba3c4439d2c25b067925059802e4e723119491fa5d1bfc585 WatchSource:0}: Error finding container 0407730e2b50638ba3c4439d2c25b067925059802e4e723119491fa5d1bfc585: Status 404 returned error can't find the container with id 0407730e2b50638ba3c4439d2c25b067925059802e4e723119491fa5d1bfc585
Sep 10 14:27:54.728409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.728376    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"]
Sep 10 14:27:54.730831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.730812    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.733111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.733089    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fc61ae9e-serving-cert\""
Sep 10 14:27:54.741209 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.741144    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"]
Sep 10 14:27:54.741730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.741575    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-metrics-certs\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.742385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.741806    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkvkb\" (UniqueName: \"kubernetes.io/projected/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-kube-api-access-vkvkb\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.742385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.741841    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-maas-api-tls\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.742385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.741933    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-openshift-service-ca\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.843357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.843257    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkvkb\" (UniqueName: \"kubernetes.io/projected/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-kube-api-access-vkvkb\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.843357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.843318    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-maas-api-tls\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.843357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.843341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-openshift-service-ca\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.843899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.843370    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-metrics-certs\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.844209 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.844189    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-openshift-service-ca\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.846294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.846219    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-maas-api-tls\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.846431 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.846329    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-metrics-certs\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.851698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.851671    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkvkb\" (UniqueName: \"kubernetes.io/projected/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-kube-api-access-vkvkb\") pod \"maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:54.917077 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:54.917046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:55.017656 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.017623    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:55.043743 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.043720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:55.236455 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.236418    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:55.315166 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.315140    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"]
Sep 10 14:27:55.317461 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:27:55.317433    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c7cc6f3_1b77_4017_b64b_ad8a1a2e761f.slice/crio-388b5ab1d45793702bd1d47af265e9e138bd371377e84d3342efd7aa4c61c456 WatchSource:0}: Error finding container 388b5ab1d45793702bd1d47af265e9e138bd371377e84d3342efd7aa4c61c456: Status 404 returned error can't find the container with id 388b5ab1d45793702bd1d47af265e9e138bd371377e84d3342efd7aa4c61c456
Sep 10 14:27:55.317597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.317578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:55.491855 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.491819    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s" event={"ID":"35779236-c2e0-4a73-b006-12028baad464","Type":"ContainerStarted","Data":"0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e"}
Sep 10 14:27:55.492045 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.491863    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s" event={"ID":"35779236-c2e0-4a73-b006-12028baad464","Type":"ContainerStarted","Data":"0407730e2b50638ba3c4439d2c25b067925059802e4e723119491fa5d1bfc585"}
Sep 10 14:27:55.493150 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.493125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl" event={"ID":"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f","Type":"ContainerStarted","Data":"388b5ab1d45793702bd1d47af265e9e138bd371377e84d3342efd7aa4c61c456"}
Sep 10 14:27:55.508911 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:55.508866    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s" podStartSLOduration=2.508852549 podStartE2EDuration="2.508852549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:27:55.507444252 +0000 UTC m=+957.595901823" watchObservedRunningTime="2026-09-10 14:27:55.508852549 +0000 UTC m=+957.597310121"
Sep 10 14:27:56.012635 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.012601    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:27:56.162429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.162407    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:56.257367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257333    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-credential-socket\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257377    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-socket\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257413    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-podinfo\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257446    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istiod-ca-cert\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257476    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4l54\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-kube-api-access-j4l54\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257499    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-token\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257563    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-certs\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257587    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-data\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.257839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257635    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:27:56.257839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257654    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-envoy\") pod \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\" (UID: \"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939\") "
Sep 10 14:27:56.258124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.257919    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.258124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.258004    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:27:56.258124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.258029    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:27:56.258347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.258320    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-data" (OuterVolumeSpecName: "istio-data") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:27:56.258879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.258849    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:27:56.260516 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.260491    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:27:56.260610 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.260534    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:27:56.260675 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.260656    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-kube-api-access-j4l54" (OuterVolumeSpecName: "kube-api-access-j4l54") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "kube-api-access-j4l54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:27:56.260713 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.260681    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-token" (OuterVolumeSpecName: "istio-token") pod "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" (UID: "116e3ad9-7af9-4c92-bb0d-ef1b8d35e939"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:27:56.358672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358636    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.358672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358666    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.358672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358675    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.358672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358683    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.359008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358690    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.359008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358700    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.359008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358707    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4l54\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-kube-api-access-j4l54\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.359008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.358716    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:27:56.500311 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.500279    2576 generic.go:356] "Generic (PLEG): container finished" podID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerID="4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf" exitCode=0
Sep 10 14:27:56.500478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.500345    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"
Sep 10 14:27:56.500478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.500358    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" event={"ID":"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939","Type":"ContainerDied","Data":"4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf"}
Sep 10 14:27:56.500478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.500422    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp" event={"ID":"116e3ad9-7af9-4c92-bb0d-ef1b8d35e939","Type":"ContainerDied","Data":"854d75a560c155f0349593afc2df05289aa8ae0346a7406f48d1a06df2695f47"}
Sep 10 14:27:56.500478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.500454    2576 scope.go:117] "RemoveContainer" containerID="4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf"
Sep 10 14:27:56.502312 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.502282    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl" event={"ID":"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f","Type":"ContainerStarted","Data":"8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58"}
Sep 10 14:27:56.509910 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.509894    2576 scope.go:117] "RemoveContainer" containerID="daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"
Sep 10 14:27:56.517982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.517966    2576 scope.go:117] "RemoveContainer" containerID="4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf"
Sep 10 14:27:56.518265 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:27:56.518225    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf\": container with ID starting with 4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf not found: ID does not exist" containerID="4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf"
Sep 10 14:27:56.518328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.518275    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf"} err="failed to get container status \"4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf\": rpc error: code = NotFound desc = could not find container \"4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf\": container with ID starting with 4f8e4dd66c2f7ec2b4558e463b59181d580cb9d7f6e4c1d493ef581a2e48eadf not found: ID does not exist"
Sep 10 14:27:56.518328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.518291    2576 scope.go:117] "RemoveContainer" containerID="daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"
Sep 10 14:27:56.518528 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:27:56.518514    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb\": container with ID starting with daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb not found: ID does not exist" containerID="daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"
Sep 10 14:27:56.518565 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.518530    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb"} err="failed to get container status \"daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb\": rpc error: code = NotFound desc = could not find container \"daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb\": container with ID starting with daedd7553af5303ab73c78f225c26a59c5dc10cbb918f8b78453682b35371bcb not found: ID does not exist"
Sep 10 14:27:56.521908 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.521872    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl" podStartSLOduration=1.753954606 podStartE2EDuration="2.521861015s" totalImagesPullingTime="767.906409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:27:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:27:56.52084581 +0000 UTC m=+958.609303380" watchObservedRunningTime="2026-09-10 14:27:56.521861015 +0000 UTC m=+958.610318635"
Sep 10 14:27:56.535889 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.535867    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"]
Sep 10 14:27:56.539252 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:56.539213    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-59ea8af5-gw-openshift-default-6f98d67796-v9dnp"]
Sep 10 14:27:57.508153 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:57.508121    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:27:58.427956 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:27:58.427928    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" path="/var/lib/kubelet/pods/116e3ad9-7af9-4c92-bb0d-ef1b8d35e939/volumes"
Sep 10 14:28:03.517563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:03.517533    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:28:04.316267 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:04.316214    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:28:04.316416 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:04.316289    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:28:04.328839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:04.328817    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:28:04.551614 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:04.551580    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:28:12.206804 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:12.206762    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:28:25.173968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.173884    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"]
Sep 10 14:28:25.174554 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.174525    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl" podUID="1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" containerName="maas-api" containerID="cri-o://8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58" gracePeriod=30
Sep 10 14:28:25.207518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.207485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:25.418783 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.418751    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:28:25.531861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.531785    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-openshift-service-ca\") pod \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") "
Sep 10 14:28:25.531861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.531854    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-maas-api-tls\") pod \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") "
Sep 10 14:28:25.532070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.531878    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-metrics-certs\") pod \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") "
Sep 10 14:28:25.532070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.531905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkvkb\" (UniqueName: \"kubernetes.io/projected/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-kube-api-access-vkvkb\") pod \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\" (UID: \"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f\") "
Sep 10 14:28:25.532200 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.532171    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" (UID: "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:28:25.534394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.534369    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" (UID: "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:28:25.534531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.534497    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" (UID: "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:28:25.534531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.534510    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-kube-api-access-vkvkb" (OuterVolumeSpecName: "kube-api-access-vkvkb") pod "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" (UID: "1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f"). InnerVolumeSpecName "kube-api-access-vkvkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:28:25.632669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.632638    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:28:25.632669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.632665    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:28:25.632669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.632675    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:28:25.632891 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.632684    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkvkb\" (UniqueName: \"kubernetes.io/projected/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f-kube-api-access-vkvkb\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:28:25.653692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.653658    2576 generic.go:356] "Generic (PLEG): container finished" podID="1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" containerID="8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58" exitCode=0
Sep 10 14:28:25.653864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.653724    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"
Sep 10 14:28:25.653864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.653753    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl" event={"ID":"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f","Type":"ContainerDied","Data":"8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58"}
Sep 10 14:28:25.653864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.653839    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl" event={"ID":"1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f","Type":"ContainerDied","Data":"388b5ab1d45793702bd1d47af265e9e138bd371377e84d3342efd7aa4c61c456"}
Sep 10 14:28:25.654017 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.653870    2576 scope.go:117] "RemoveContainer" containerID="8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58"
Sep 10 14:28:25.663403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.663384    2576 scope.go:117] "RemoveContainer" containerID="8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58"
Sep 10 14:28:25.663716 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:28:25.663696    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58\": container with ID starting with 8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58 not found: ID does not exist" containerID="8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58"
Sep 10 14:28:25.663777 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.663729    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58"} err="failed to get container status \"8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58\": rpc error: code = NotFound desc = could not find container \"8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58\": container with ID starting with 8f0f4ce91a27b019ee22e6fdce80e2a9ef4a0a56d58a2b20733a73889579bd58 not found: ID does not exist"
Sep 10 14:28:25.674972 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.674947    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"]
Sep 10 14:28:25.679152 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:25.679130    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fc61ae9e-74786d56d7-kwmdl"]
Sep 10 14:28:26.428033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:26.428001    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" path="/var/lib/kubelet/pods/1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f/volumes"
Sep 10 14:28:26.525906 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:26.525875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:26.917700 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:26.917624    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.169840 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.169812    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"]
Sep 10 14:28:27.170358 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170341    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" containerName="maas-api"
Sep 10 14:28:27.170445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170360    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" containerName="maas-api"
Sep 10 14:28:27.170445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170375    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy"
Sep 10 14:28:27.170445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170384    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy"
Sep 10 14:28:27.170445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170394    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy"
Sep 10 14:28:27.170445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170402    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy"
Sep 10 14:28:27.170601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170502    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy"
Sep 10 14:28:27.170601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170513    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="116e3ad9-7af9-4c92-bb0d-ef1b8d35e939" containerName="istio-proxy"
Sep 10 14:28:27.170601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.170524    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c7cc6f3-1b77-4017-b64b-ad8a1a2e761f" containerName="maas-api"
Sep 10 14:28:27.173725 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.173700    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.176616 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.176580    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-369894e2-openshift-default-dockercfg-q766t\""
Sep 10 14:28:27.186764 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.186742    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"]
Sep 10 14:28:27.203300 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.203267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.348392 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-credential-socket\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348552 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348398    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-token\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348552 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348421    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-envoy\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348552 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348445    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-data\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348552 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348525    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-socket\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-podinfo\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348585    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istiod-ca-cert\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348637    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmt6m\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-kube-api-access-rmt6m\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.348742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.348666    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-certs\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.395594 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.395559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:27.449621 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449548    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-certs\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.449621 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449602    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-credential-socket\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449626    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-token\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-envoy\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-data\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-socket\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-podinfo\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449844    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istiod-ca-cert\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449909    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmt6m\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-kube-api-access-rmt6m\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.449989    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-certs\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.450026    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-data\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.450112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-credential-socket\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.450454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-socket\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.450608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.450578    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istiod-ca-cert\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.452859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.452830    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-envoy\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.453145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.453126    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-podinfo\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.458074 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.458055    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmt6m\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-kube-api-access-rmt6m\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.458180 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.458121    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-token\") pod \"e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") " pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.485623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.485599    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:27.761754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.761729    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"]
Sep 10 14:28:27.764009 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:28:27.763979    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeafb7513_4576_46e7_ab7c_e3a8030ab45b.slice/crio-75b47515e372deaefb741fa90e79ffea8531f4ca623472e44d14372a66c5920b WatchSource:0}: Error finding container 75b47515e372deaefb741fa90e79ffea8531f4ca623472e44d14372a66c5920b: Status 404 returned error can't find the container with id 75b47515e372deaefb741fa90e79ffea8531f4ca623472e44d14372a66c5920b
Sep 10 14:28:27.766408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.766370    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:28:27.766534 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.766460    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:28:27.766534 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:27.766505    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:28:28.263110 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:28.263067    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:28:28.671422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:28.671395    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" event={"ID":"eafb7513-4576-46e7-ab7c-e3a8030ab45b","Type":"ContainerStarted","Data":"f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"}
Sep 10 14:28:28.671948 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:28.671457    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" event={"ID":"eafb7513-4576-46e7-ab7c-e3a8030ab45b","Type":"ContainerStarted","Data":"75b47515e372deaefb741fa90e79ffea8531f4ca623472e44d14372a66c5920b"}
Sep 10 14:28:28.693488 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:28.693438    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podStartSLOduration=1.6934228249999999 podStartE2EDuration="1.693422825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:28.691115214 +0000 UTC m=+990.779572784" watchObservedRunningTime="2026-09-10 14:28:28.693422825 +0000 UTC m=+990.781880396"
Sep 10 14:28:29.486029 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:29.485992    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:29.491049 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:29.491020    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:29.676606 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:29.676579    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:29.677673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:29.677653    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:30.005289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:30.005252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:30.105143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:30.105100    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:31.109179 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.109147    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"]
Sep 10 14:28:31.112353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.112327    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.114698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.114674    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-369894e2-serving-cert\""
Sep 10 14:28:31.124845 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.124818    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"]
Sep 10 14:28:31.148441 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.148406    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:31.253832 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.253798    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:31.292008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.291965    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-maas-api-tls\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.292008 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.292025    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfhsn\" (UniqueName: \"kubernetes.io/projected/f3b48a43-f24c-4f12-89bd-fd9f003a0271-kube-api-access-rfhsn\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.292289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.292115    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-metrics-certs\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.292289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.292180    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b48a43-f24c-4f12-89bd-fd9f003a0271-openshift-service-ca\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.356508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.356472    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:31.393741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.393654    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-metrics-certs\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.393741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.393711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b48a43-f24c-4f12-89bd-fd9f003a0271-openshift-service-ca\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.393932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.393748    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-maas-api-tls\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.393932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.393771    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfhsn\" (UniqueName: \"kubernetes.io/projected/f3b48a43-f24c-4f12-89bd-fd9f003a0271-kube-api-access-rfhsn\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.394443 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.394421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b48a43-f24c-4f12-89bd-fd9f003a0271-openshift-service-ca\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.396366 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.396332    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-maas-api-tls\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.396473 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.396369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-metrics-certs\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.402001 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.401968    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfhsn\" (UniqueName: \"kubernetes.io/projected/f3b48a43-f24c-4f12-89bd-fd9f003a0271-kube-api-access-rfhsn\") pod \"maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.427589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.427563    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:31.616363 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.616307    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:31.699881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.699855    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"]
Sep 10 14:28:31.702546 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:28:31.702512    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3b48a43_f24c_4f12_89bd_fd9f003a0271.slice/crio-c508b8beb569f3b62d874d0ef445a543b8bbb73a4bc753c420754fae13334b1f WatchSource:0}: Error finding container c508b8beb569f3b62d874d0ef445a543b8bbb73a4bc753c420754fae13334b1f: Status 404 returned error can't find the container with id c508b8beb569f3b62d874d0ef445a543b8bbb73a4bc753c420754fae13334b1f
Sep 10 14:28:31.703378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:31.703317    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:32.696402 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:32.696366    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf" event={"ID":"f3b48a43-f24c-4f12-89bd-fd9f003a0271","Type":"ContainerStarted","Data":"8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3"}
Sep 10 14:28:32.696786 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:32.696425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf" event={"ID":"f3b48a43-f24c-4f12-89bd-fd9f003a0271","Type":"ContainerStarted","Data":"c508b8beb569f3b62d874d0ef445a543b8bbb73a4bc753c420754fae13334b1f"}
Sep 10 14:28:32.696786 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:32.696540    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:32.716553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:32.716500    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf" podStartSLOduration=1.28041815 podStartE2EDuration="1.716485303s" totalImagesPullingTime="436.067153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:28:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:28:32.714442911 +0000 UTC m=+994.802900482" watchObservedRunningTime="2026-09-10 14:28:32.716485303 +0000 UTC m=+994.804942873"
Sep 10 14:28:38.707475 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:38.707441    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:28:40.537956 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:40.537885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:45.206068 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:45.206022    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:28:45.718438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:45.718396    2576 patch_prober.go:28] interesting pod/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 14:28:45.718661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:45.718478    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 14:28:47.776341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:47.776309    2576 generic.go:356] "Generic (PLEG): container finished" podID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerID="f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592" exitCode=0
Sep 10 14:28:47.776764 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:47.776391    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" event={"ID":"eafb7513-4576-46e7-ab7c-e3a8030ab45b","Type":"ContainerDied","Data":"f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"}
Sep 10 14:28:47.776818 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:47.776802    2576 scope.go:117] "RemoveContainer" containerID="f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"
Sep 10 14:28:47.777929 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:47.777901    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:28:47.777997 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:47.777982    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:28:47.778034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:47.778018    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:28:48.784502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:48.784458    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" event={"ID":"eafb7513-4576-46e7-ab7c-e3a8030ab45b","Type":"ContainerStarted","Data":"78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c"}
Sep 10 14:28:49.485955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:49.485916    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:49.486165 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:49.486137    2576 patch_prober.go:28] interesting pod/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 10 14:28:49.486267 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:49.486203    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 10 14:28:50.486853 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:50.486816    2576 patch_prober.go:28] interesting pod/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 10 14:28:50.487280 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:50.486877    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 10 14:28:50.674996 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:50.674926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:51.485939 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:51.485909    2576 patch_prober.go:28] interesting pod/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 10 14:28:51.486119 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:51.485974    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 10 14:28:52.136780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:52.136748    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:52.486408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:52.486374    2576 patch_prober.go:28] interesting pod/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 10 14:28:52.486602 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:52.486448    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 10 14:28:53.006199 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:53.006154    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"]
Sep 10 14:28:53.025581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:53.025550    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:53.486511 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:53.486474    2576 patch_prober.go:28] interesting pod/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Sep 10 14:28:53.486886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:53.486543    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Sep 10 14:28:54.489832 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:54.489803    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:54.490192 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:54.490168    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:54.491045 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:54.491026    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:28:58.025224 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:58.025191    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:58.711118 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:28:58.711087    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:29:00.123340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:00.123303    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"]
Sep 10 14:29:00.134820 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:00.134794    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:01.589639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.589600    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"]
Sep 10 14:29:01.590111 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.589916    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf" podUID="f3b48a43-f24c-4f12-89bd-fd9f003a0271" containerName="maas-api" containerID="cri-o://8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3" gracePeriod=30
Sep 10 14:29:01.616771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.616740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:01.682834 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.682803    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:01.782772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.782733    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:01.838144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.838117    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:29:01.846068 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.845945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:01.851804 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.851774    2576 generic.go:356] "Generic (PLEG): container finished" podID="f3b48a43-f24c-4f12-89bd-fd9f003a0271" containerID="8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3" exitCode=0
Sep 10 14:29:01.851978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.851831    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"
Sep 10 14:29:01.851978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.851864    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf" event={"ID":"f3b48a43-f24c-4f12-89bd-fd9f003a0271","Type":"ContainerDied","Data":"8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3"}
Sep 10 14:29:01.851978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.851926    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf" event={"ID":"f3b48a43-f24c-4f12-89bd-fd9f003a0271","Type":"ContainerDied","Data":"c508b8beb569f3b62d874d0ef445a543b8bbb73a4bc753c420754fae13334b1f"}
Sep 10 14:29:01.851978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.851960    2576 scope.go:117] "RemoveContainer" containerID="8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3"
Sep 10 14:29:01.864249 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.864203    2576 scope.go:117] "RemoveContainer" containerID="8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3"
Sep 10 14:29:01.864659 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:01.864636    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3\": container with ID starting with 8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3 not found: ID does not exist" containerID="8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3"
Sep 10 14:29:01.864770 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.864666    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3"} err="failed to get container status \"8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3\": rpc error: code = NotFound desc = could not find container \"8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3\": container with ID starting with 8a19028e9658fd13af352dbaee8bbb637d849ee35308f2d3c5e3101e2c416ae3 not found: ID does not exist"
Sep 10 14:29:01.911005 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.910972    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"]
Sep 10 14:29:01.911681 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.911246    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy" containerID="cri-o://78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c" gracePeriod=30
Sep 10 14:29:01.928100 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.928072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:01.993508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.993475    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-maas-api-tls\") pod \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") "
Sep 10 14:29:01.993668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.993532    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b48a43-f24c-4f12-89bd-fd9f003a0271-openshift-service-ca\") pod \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") "
Sep 10 14:29:01.993668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.993618    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-metrics-certs\") pod \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") "
Sep 10 14:29:01.993668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.993663    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfhsn\" (UniqueName: \"kubernetes.io/projected/f3b48a43-f24c-4f12-89bd-fd9f003a0271-kube-api-access-rfhsn\") pod \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\" (UID: \"f3b48a43-f24c-4f12-89bd-fd9f003a0271\") "
Sep 10 14:29:01.993957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.993925    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f3b48a43-f24c-4f12-89bd-fd9f003a0271-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f3b48a43-f24c-4f12-89bd-fd9f003a0271" (UID: "f3b48a43-f24c-4f12-89bd-fd9f003a0271"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:01.995926 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.995897    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f3b48a43-f24c-4f12-89bd-fd9f003a0271" (UID: "f3b48a43-f24c-4f12-89bd-fd9f003a0271"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:01.995926 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.995910    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3b48a43-f24c-4f12-89bd-fd9f003a0271-kube-api-access-rfhsn" (OuterVolumeSpecName: "kube-api-access-rfhsn") pod "f3b48a43-f24c-4f12-89bd-fd9f003a0271" (UID: "f3b48a43-f24c-4f12-89bd-fd9f003a0271"). InnerVolumeSpecName "kube-api-access-rfhsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:01.996042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:01.995930    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f3b48a43-f24c-4f12-89bd-fd9f003a0271" (UID: "f3b48a43-f24c-4f12-89bd-fd9f003a0271"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:02.095393 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.095354    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:02.095393 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.095385    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfhsn\" (UniqueName: \"kubernetes.io/projected/f3b48a43-f24c-4f12-89bd-fd9f003a0271-kube-api-access-rfhsn\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:02.095393 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.095396    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f3b48a43-f24c-4f12-89bd-fd9f003a0271-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:02.095393 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.095404    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f3b48a43-f24c-4f12-89bd-fd9f003a0271-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:02.172858 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.172828    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"]
Sep 10 14:29:02.176281 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.176255    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-369894e2-7b58cb8fc8-8tlsf"]
Sep 10 14:29:02.427882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.427848    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3b48a43-f24c-4f12-89bd-fd9f003a0271" path="/var/lib/kubelet/pods/f3b48a43-f24c-4f12-89bd-fd9f003a0271/volumes"
Sep 10 14:29:02.554675 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.554635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.651718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.651681    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:02.757869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:02.757782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:04.146160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:04.146132    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:04.239266 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:04.239217    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:04.357058 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:04.357023    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:07.159732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.159707    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:29:07.240573 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240480    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmt6m\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-kube-api-access-rmt6m\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240573 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240520    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-token\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240573 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240543    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-envoy\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240609    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-socket\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240660    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-data\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240728    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-certs\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240808    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-credential-socket\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.240852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istiod-ca-cert\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.241088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240867    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-podinfo\") pod \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\" (UID: \"eafb7513-4576-46e7-ab7c-e3a8030ab45b\") "
Sep 10 14:29:07.241088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240895    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:07.241088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.240950    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-data" (OuterVolumeSpecName: "istio-data") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:07.241088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.241060    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:07.241487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.241375    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:07.241487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.241362    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.241487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.241447    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.241487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.241462    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.241869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.241665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:07.242930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.242905    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-kube-api-access-rmt6m" (OuterVolumeSpecName: "kube-api-access-rmt6m") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "kube-api-access-rmt6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:07.243029 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.242926    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-token" (OuterVolumeSpecName: "istio-token") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:07.243029 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.242975    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:07.243105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.243093    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eafb7513-4576-46e7-ab7c-e3a8030ab45b" (UID: "eafb7513-4576-46e7-ab7c-e3a8030ab45b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:29:07.342718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.342681    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmt6m\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-kube-api-access-rmt6m\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.342718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.342714    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.342718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.342724    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.342718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.342732    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eafb7513-4576-46e7-ab7c-e3a8030ab45b-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.342988 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.342740    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.342988 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.342748    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eafb7513-4576-46e7-ab7c-e3a8030ab45b-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:07.885718 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.885678    2576 generic.go:356] "Generic (PLEG): container finished" podID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerID="78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c" exitCode=0
Sep 10 14:29:07.885890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.885725    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"
Sep 10 14:29:07.885890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.885737    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" event={"ID":"eafb7513-4576-46e7-ab7c-e3a8030ab45b","Type":"ContainerDied","Data":"78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c"}
Sep 10 14:29:07.885890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.885789    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt" event={"ID":"eafb7513-4576-46e7-ab7c-e3a8030ab45b","Type":"ContainerDied","Data":"75b47515e372deaefb741fa90e79ffea8531f4ca623472e44d14372a66c5920b"}
Sep 10 14:29:07.885890 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.885814    2576 scope.go:117] "RemoveContainer" containerID="78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c"
Sep 10 14:29:07.895503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.895485    2576 scope.go:117] "RemoveContainer" containerID="f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"
Sep 10 14:29:07.904172 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.904151    2576 scope.go:117] "RemoveContainer" containerID="78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c"
Sep 10 14:29:07.904534 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:07.904514    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c\": container with ID starting with 78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c not found: ID does not exist" containerID="78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c"
Sep 10 14:29:07.904602 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.904547    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c"} err="failed to get container status \"78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c\": rpc error: code = NotFound desc = could not find container \"78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c\": container with ID starting with 78923d9ebf7e8e457dc28efdba360478d2859fe2425754737472809847dc0b6c not found: ID does not exist"
Sep 10 14:29:07.904602 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.904572    2576 scope.go:117] "RemoveContainer" containerID="f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"
Sep 10 14:29:07.904851 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:07.904829    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592\": container with ID starting with f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592 not found: ID does not exist" containerID="f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"
Sep 10 14:29:07.904900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.904858    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592"} err="failed to get container status \"f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592\": rpc error: code = NotFound desc = could not find container \"f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592\": container with ID starting with f463c47c5ed288a6138c8571fcfe0f2a65a317e5186a04831401b226b3b3a592 not found: ID does not exist"
Sep 10 14:29:07.914456 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.914437    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"]
Sep 10 14:29:07.918714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:07.918692    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-369894e2-openshift-default-67dcc9655b-q75mt"]
Sep 10 14:29:08.026785 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.026734    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z" podUID="4da36acb-99a6-4866-9072-8f502a41215c" containerName="main" containerID="cri-o://c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d" gracePeriod=45
Sep 10 14:29:08.284959 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.284935    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:29:08.427801 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.427773    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" path="/var/lib/kubelet/pods/eafb7513-4576-46e7-ab7c-e3a8030ab45b/volumes"
Sep 10 14:29:08.454218 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454188    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4da36acb-99a6-4866-9072-8f502a41215c-tls-certs\") pod \"4da36acb-99a6-4866-9072-8f502a41215c\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") "
Sep 10 14:29:08.454384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454281    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-model-cache\") pod \"4da36acb-99a6-4866-9072-8f502a41215c\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") "
Sep 10 14:29:08.454384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454343    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-tmp-dir\") pod \"4da36acb-99a6-4866-9072-8f502a41215c\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") "
Sep 10 14:29:08.454537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454495    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-dshm\") pod \"4da36acb-99a6-4866-9072-8f502a41215c\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") "
Sep 10 14:29:08.454622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454565    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-home\") pod \"4da36acb-99a6-4866-9072-8f502a41215c\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") "
Sep 10 14:29:08.454622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454562    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4da36acb-99a6-4866-9072-8f502a41215c" (UID: "4da36acb-99a6-4866-9072-8f502a41215c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:08.454622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454602    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xd72q\" (UniqueName: \"kubernetes.io/projected/4da36acb-99a6-4866-9072-8f502a41215c-kube-api-access-xd72q\") pod \"4da36acb-99a6-4866-9072-8f502a41215c\" (UID: \"4da36acb-99a6-4866-9072-8f502a41215c\") "
Sep 10 14:29:08.454622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454577    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4da36acb-99a6-4866-9072-8f502a41215c" (UID: "4da36acb-99a6-4866-9072-8f502a41215c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:08.454815 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454739    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-home" (OuterVolumeSpecName: "home") pod "4da36acb-99a6-4866-9072-8f502a41215c" (UID: "4da36acb-99a6-4866-9072-8f502a41215c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:08.455037 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454841    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:08.455037 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454861    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:08.455037 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.454877    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:08.456712 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.456685    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4da36acb-99a6-4866-9072-8f502a41215c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4da36acb-99a6-4866-9072-8f502a41215c" (UID: "4da36acb-99a6-4866-9072-8f502a41215c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:08.456712 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.456705    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4da36acb-99a6-4866-9072-8f502a41215c-kube-api-access-xd72q" (OuterVolumeSpecName: "kube-api-access-xd72q") pod "4da36acb-99a6-4866-9072-8f502a41215c" (UID: "4da36acb-99a6-4866-9072-8f502a41215c"). InnerVolumeSpecName "kube-api-access-xd72q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:08.456853 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.456686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-dshm" (OuterVolumeSpecName: "dshm") pod "4da36acb-99a6-4866-9072-8f502a41215c" (UID: "4da36acb-99a6-4866-9072-8f502a41215c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:08.555454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.555423    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4da36acb-99a6-4866-9072-8f502a41215c-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:08.555454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.555453    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xd72q\" (UniqueName: \"kubernetes.io/projected/4da36acb-99a6-4866-9072-8f502a41215c-kube-api-access-xd72q\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:08.555673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.555470    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4da36acb-99a6-4866-9072-8f502a41215c-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:08.892142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.892049    2576 generic.go:356] "Generic (PLEG): container finished" podID="4da36acb-99a6-4866-9072-8f502a41215c" containerID="c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d" exitCode=0
Sep 10 14:29:08.892142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.892099    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"
Sep 10 14:29:08.892406 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.892139    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z" event={"ID":"4da36acb-99a6-4866-9072-8f502a41215c","Type":"ContainerDied","Data":"c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d"}
Sep 10 14:29:08.892406 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.892180    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z" event={"ID":"4da36acb-99a6-4866-9072-8f502a41215c","Type":"ContainerDied","Data":"9fc6dd617956c7baea3a58c2797a33df20f800c50c0c5021e5cdeb4d3a228792"}
Sep 10 14:29:08.892406 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.892203    2576 scope.go:117] "RemoveContainer" containerID="c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d"
Sep 10 14:29:08.901507 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.901490    2576 scope.go:117] "RemoveContainer" containerID="c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d"
Sep 10 14:29:08.901776 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:08.901761    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d\": container with ID starting with c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d not found: ID does not exist" containerID="c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d"
Sep 10 14:29:08.901819 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.901783    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d"} err="failed to get container status \"c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d\": rpc error: code = NotFound desc = could not find container \"c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d\": container with ID starting with c42def26d989f0ef1e4fe0c5d0c64f64ed715cb8d7b2302e66c773ed4947490d not found: ID does not exist"
Sep 10 14:29:08.913918 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.913889    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"]
Sep 10 14:29:08.917451 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:08.917426    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-a-9a074211-kserve-8d5ff65fb-tz89z"]
Sep 10 14:29:10.154544 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:10.154509    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:10.259502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:10.259465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:10.358731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:10.358700    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:10.426967 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:10.426935    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4da36acb-99a6-4866-9072-8f502a41215c" path="/var/lib/kubelet/pods/4da36acb-99a6-4866-9072-8f502a41215c/volumes"
Sep 10 14:29:10.455556 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:10.455525    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:10.837664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:10.837574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:13.812549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:13.812510    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:29:14.735816 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:14.735782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:14.833144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:14.833118    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:14.902674 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:14.902643    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:14.962854 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:14.962825    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:15.143459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.143352    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s" podUID="35779236-c2e0-4a73-b006-12028baad464" containerName="main" containerID="cri-o://0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e" gracePeriod=45
Sep 10 14:29:15.338809 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.338774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:15.391937 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.391914    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:29:15.418817 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.418748    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-dshm\") pod \"35779236-c2e0-4a73-b006-12028baad464\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") "
Sep 10 14:29:15.418817 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.418794    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w878c\" (UniqueName: \"kubernetes.io/projected/35779236-c2e0-4a73-b006-12028baad464-kube-api-access-w878c\") pod \"35779236-c2e0-4a73-b006-12028baad464\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") "
Sep 10 14:29:15.419013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.418835    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-tmp-dir\") pod \"35779236-c2e0-4a73-b006-12028baad464\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") "
Sep 10 14:29:15.419013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.418900    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-home\") pod \"35779236-c2e0-4a73-b006-12028baad464\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") "
Sep 10 14:29:15.419013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.418961    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35779236-c2e0-4a73-b006-12028baad464-tls-certs\") pod \"35779236-c2e0-4a73-b006-12028baad464\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") "
Sep 10 14:29:15.419013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.418994    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-model-cache\") pod \"35779236-c2e0-4a73-b006-12028baad464\" (UID: \"35779236-c2e0-4a73-b006-12028baad464\") "
Sep 10 14:29:15.419217 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.419187    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35779236-c2e0-4a73-b006-12028baad464" (UID: "35779236-c2e0-4a73-b006-12028baad464"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:15.419287 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.419199    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-home" (OuterVolumeSpecName: "home") pod "35779236-c2e0-4a73-b006-12028baad464" (UID: "35779236-c2e0-4a73-b006-12028baad464"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:15.419338 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.419307    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.419338 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.419326    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.419418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.419339    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-model-cache" (OuterVolumeSpecName: "model-cache") pod "35779236-c2e0-4a73-b006-12028baad464" (UID: "35779236-c2e0-4a73-b006-12028baad464"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:15.421105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.421082    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-dshm" (OuterVolumeSpecName: "dshm") pod "35779236-c2e0-4a73-b006-12028baad464" (UID: "35779236-c2e0-4a73-b006-12028baad464"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:15.421206 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.421106    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35779236-c2e0-4a73-b006-12028baad464-kube-api-access-w878c" (OuterVolumeSpecName: "kube-api-access-w878c") pod "35779236-c2e0-4a73-b006-12028baad464" (UID: "35779236-c2e0-4a73-b006-12028baad464"). InnerVolumeSpecName "kube-api-access-w878c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:15.421579 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.421558    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35779236-c2e0-4a73-b006-12028baad464-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35779236-c2e0-4a73-b006-12028baad464" (UID: "35779236-c2e0-4a73-b006-12028baad464"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:29:15.520319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.520280    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35779236-c2e0-4a73-b006-12028baad464-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.520319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.520310    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.520319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.520322    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35779236-c2e0-4a73-b006-12028baad464-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.520558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.520334    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w878c\" (UniqueName: \"kubernetes.io/projected/35779236-c2e0-4a73-b006-12028baad464-kube-api-access-w878c\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:15.928777 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.928742    2576 generic.go:356] "Generic (PLEG): container finished" podID="35779236-c2e0-4a73-b006-12028baad464" containerID="0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e" exitCode=0
Sep 10 14:29:15.929277 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.928799    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"
Sep 10 14:29:15.929277 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.928828    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s" event={"ID":"35779236-c2e0-4a73-b006-12028baad464","Type":"ContainerDied","Data":"0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e"}
Sep 10 14:29:15.929277 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.928876    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s" event={"ID":"35779236-c2e0-4a73-b006-12028baad464","Type":"ContainerDied","Data":"0407730e2b50638ba3c4439d2c25b067925059802e4e723119491fa5d1bfc585"}
Sep 10 14:29:15.929277 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.928894    2576 scope.go:117] "RemoveContainer" containerID="0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e"
Sep 10 14:29:15.938277 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.938263    2576 scope.go:117] "RemoveContainer" containerID="0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e"
Sep 10 14:29:15.938515 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:15.938497    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e\": container with ID starting with 0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e not found: ID does not exist" containerID="0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e"
Sep 10 14:29:15.938625 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.938523    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e"} err="failed to get container status \"0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e\": rpc error: code = NotFound desc = could not find container \"0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e\": container with ID starting with 0f32ef9a9ff2ba64c77922b1aa4b503d4ada7dc886d824ea60df66a00f4bc32e not found: ID does not exist"
Sep 10 14:29:15.949416 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.949393    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"]
Sep 10 14:29:15.953741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:15.953718    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-conflict-b-9a074211-kserve-65c99c47c8-s766s"]
Sep 10 14:29:16.428132 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:16.428096    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35779236-c2e0-4a73-b006-12028baad464" path="/var/lib/kubelet/pods/35779236-c2e0-4a73-b006-12028baad464/volumes"
Sep 10 14:29:21.339261 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.339194    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:21.398034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.398000    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:21.570986 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.570952    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"]
Sep 10 14:29:21.571276 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.571201    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" containerID="cri-o://c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331" gracePeriod=30
Sep 10 14:29:21.594626 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.594562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:21.833490 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.833452    2576 patch_prober.go:28] interesting pod/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 14:29:21.833644 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:21.833515    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:29:22.012856 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.012823    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:22.306382 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306309    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"]
Sep 10 14:29:22.306708 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306696    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35779236-c2e0-4a73-b006-12028baad464" containerName="main"
Sep 10 14:29:22.306754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306710    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="35779236-c2e0-4a73-b006-12028baad464" containerName="main"
Sep 10 14:29:22.306754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306725    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3b48a43-f24c-4f12-89bd-fd9f003a0271" containerName="maas-api"
Sep 10 14:29:22.306754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306731    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3b48a43-f24c-4f12-89bd-fd9f003a0271" containerName="maas-api"
Sep 10 14:29:22.306754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306742    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4da36acb-99a6-4866-9072-8f502a41215c" containerName="main"
Sep 10 14:29:22.306754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306748    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="4da36acb-99a6-4866-9072-8f502a41215c" containerName="main"
Sep 10 14:29:22.306754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306755    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306760    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306769    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306774    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306838    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="4da36acb-99a6-4866-9072-8f502a41215c" containerName="main"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306846    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306854    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="35779236-c2e0-4a73-b006-12028baad464" containerName="main"
Sep 10 14:29:22.306930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.306860    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3b48a43-f24c-4f12-89bd-fd9f003a0271" containerName="maas-api"
Sep 10 14:29:22.309599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.309583    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.312065 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.312043    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-ba183c-openshift-default-dockercfg-4k7cj\""
Sep 10 14:29:22.322998 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.322974    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"]
Sep 10 14:29:22.328053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.328026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:22.380669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380639    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-podinfo\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380676    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-certs\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380693    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-envoy\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380725    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-credential-socket\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380749    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-token\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380772    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-socket\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380793    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istiod-ca-cert\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-data\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.381021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.380862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q92l5\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-kube-api-access-q92l5\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.424627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.424598    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:22.481673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481636    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q92l5\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-kube-api-access-q92l5\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.481835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481715    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-podinfo\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.481835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481744    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-certs\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.481835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481768    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-envoy\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.481835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481797    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-credential-socket\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.481835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481824    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-token\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482140 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481855    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-socket\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482140 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481896    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istiod-ca-cert\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482140 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.481956    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-data\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.482177    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-certs\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.482257    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-socket\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.482306    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-data\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482456 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.482354    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-credential-socket\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.482774 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.482739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istiod-ca-cert\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.484384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.484353    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-envoy\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.484570 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.484553    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-podinfo\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.489218 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.489192    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-token\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.489449 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.489434    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q92l5\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-kube-api-access-q92l5\") pod \"e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") " pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.620281 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.620168    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:22.896188 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.896160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"]
Sep 10 14:29:22.898050 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:29:22.898023    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52dacbfe_8c9f_4fe5_8f84_a0c3eec4de9b.slice/crio-36782c251b95860a4cec52d6893bd6a9d41de7633db32cc6ba61bc5c721d7a1f WatchSource:0}: Error finding container 36782c251b95860a4cec52d6893bd6a9d41de7633db32cc6ba61bc5c721d7a1f: Status 404 returned error can't find the container with id 36782c251b95860a4cec52d6893bd6a9d41de7633db32cc6ba61bc5c721d7a1f
Sep 10 14:29:22.900247 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.900197    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:29:22.900347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.900296    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:29:22.900347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.900328    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:29:22.969174 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.969140    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" event={"ID":"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b","Type":"ContainerStarted","Data":"8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"}
Sep 10 14:29:22.969351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.969190    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" event={"ID":"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b","Type":"ContainerStarted","Data":"36782c251b95860a4cec52d6893bd6a9d41de7633db32cc6ba61bc5c721d7a1f"}
Sep 10 14:29:22.989360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:22.989301    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podStartSLOduration=0.989274599 podStartE2EDuration="989.274599ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:22.987266949 +0000 UTC m=+1045.075724526" watchObservedRunningTime="2026-09-10 14:29:22.989274599 +0000 UTC m=+1045.077732235"
Sep 10 14:29:23.607385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:23.607352    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:29:23.620878 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:23.620850    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:23.626034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:23.626002    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:23.974178 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:23.974146    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:23.975089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:23.975074    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:24.879800 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:24.879768    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:25.006588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:25.006526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.126577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.122750    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"]
Sep 10 14:29:26.126577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.123783    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="eafb7513-4576-46e7-ab7c-e3a8030ab45b" containerName="istio-proxy"
Sep 10 14:29:26.134791 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.134762    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.135536 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.135467    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"]
Sep 10 14:29:26.140093 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.140066    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-ba183c-serving-cert\""
Sep 10 14:29:26.154347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.154318    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.213985 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.213947    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-metrics-certs\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.214156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.213999    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-openshift-service-ca\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.214156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.214030    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.214309 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.214155    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjxdt\" (UniqueName: \"kubernetes.io/projected/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-kube-api-access-wjxdt\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.251622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.251592    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.314717 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.314672    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjxdt\" (UniqueName: \"kubernetes.io/projected/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-kube-api-access-wjxdt\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.314930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.314763    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-metrics-certs\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.314930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.314791    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-openshift-service-ca\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.314930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.314809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.314930 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:26.314911    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-serving-cert: secret "maas-api-e2e-explicit-ba183c-serving-cert" not found
Sep 10 14:29:26.315154 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:26.314970    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls podName:63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b nodeName:}" failed. No retries permitted until 2026-09-10 14:29:26.81495166 +0000 UTC m=+1048.903409222 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls") pod "maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" (UID: "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b") : secret "maas-api-e2e-explicit-ba183c-serving-cert" not found
Sep 10 14:29:26.315584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.315562    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-openshift-service-ca\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.317929 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.317905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-metrics-certs\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.324034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.324000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjxdt\" (UniqueName: \"kubernetes.io/projected/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-kube-api-access-wjxdt\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.530526 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.530485    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.631196 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.631164    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.702703 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.702664    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:26.820177 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.820144    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.823902 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.823877    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls\") pod \"maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:26.837353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.837334    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:29:26.921581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921548    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvt65\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-kube-api-access-bvt65\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.921771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921592    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-podinfo\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.921771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921611    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-certs\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.921771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921624    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-envoy\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.921771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921650    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-credential-socket\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.921771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921714    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-socket\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.922041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921782    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-token\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.922041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921837    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istiod-ca-cert\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.922041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921897    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-data\") pod \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\" (UID: \"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2\") "
Sep 10 14:29:26.922041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921942    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:26.922041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.921960    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:26.922041 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.922021    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:26.922376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.922187    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-data" (OuterVolumeSpecName: "istio-data") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:26.922376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.922307    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:29:26.922488 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.922474    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:26.922536 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.922497    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:26.922536 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.922512    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:26.923973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.923951    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:29:26.923973 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.923965    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-kube-api-access-bvt65" (OuterVolumeSpecName: "kube-api-access-bvt65") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "kube-api-access-bvt65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:26.924541 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.924523    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-token" (OuterVolumeSpecName: "istio-token") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:29:26.924607 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.924531    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" (UID: "c04d23d2-40a6-4666-bd5e-3dc89cf52ad2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:29:26.989771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.989737    2576 generic.go:356] "Generic (PLEG): container finished" podID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerID="c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331" exitCode=0
Sep 10 14:29:26.989951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.989788    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"
Sep 10 14:29:26.989951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.989797    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" event={"ID":"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2","Type":"ContainerDied","Data":"c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331"}
Sep 10 14:29:26.989951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.989861    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg" event={"ID":"c04d23d2-40a6-4666-bd5e-3dc89cf52ad2","Type":"ContainerDied","Data":"94c350575f4ae4e8ffdc8e98a4779292d4cf087fa095d330c8df0696264cb272"}
Sep 10 14:29:26.989951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:26.989892    2576 scope.go:117] "RemoveContainer" containerID="c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331"
Sep 10 14:29:27.000863 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.000837    2576 scope.go:117] "RemoveContainer" containerID="f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"
Sep 10 14:29:27.010276 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.010251    2576 scope.go:117] "RemoveContainer" containerID="c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331"
Sep 10 14:29:27.010531 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:27.010513    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331\": container with ID starting with c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331 not found: ID does not exist" containerID="c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331"
Sep 10 14:29:27.010588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.010539    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331"} err="failed to get container status \"c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331\": rpc error: code = NotFound desc = could not find container \"c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331\": container with ID starting with c2d8932d5a485740aacc728798faf2f8e98bf95c5f96222b38043e516882d331 not found: ID does not exist"
Sep 10 14:29:27.010588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.010554    2576 scope.go:117] "RemoveContainer" containerID="f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"
Sep 10 14:29:27.010805 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:29:27.010787    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d\": container with ID starting with f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d not found: ID does not exist" containerID="f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"
Sep 10 14:29:27.010843 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.010813    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d"} err="failed to get container status \"f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d\": rpc error: code = NotFound desc = could not find container \"f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d\": container with ID starting with f1b914986221f55d802196aef4e2664e1c8c1e5b602f0d2a66fad6ba4152a63d not found: ID does not exist"
Sep 10 14:29:27.011571 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.011551    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"]
Sep 10 14:29:27.015222 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.015195    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-44e3af-openshift-default-85fc544bb4-r4dpg"]
Sep 10 14:29:27.023770 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.023740    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:27.023770 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.023761    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvt65\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-kube-api-access-bvt65\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:27.023770 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.023772    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:27.023927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.023782    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:27.023927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.023789    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:27.023927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.023797    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:29:27.049605 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.049581    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:27.321484 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.321458    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"]
Sep 10 14:29:27.322741 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:29:27.322708    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b8a2a3_7d96_4f8c_b4f9_7dee0a94b11b.slice/crio-48236f7f715f5b0d2defbeabff0ae32df2c9395a7664a73791ac2246fd20a5ed WatchSource:0}: Error finding container 48236f7f715f5b0d2defbeabff0ae32df2c9395a7664a73791ac2246fd20a5ed: Status 404 returned error can't find the container with id 48236f7f715f5b0d2defbeabff0ae32df2c9395a7664a73791ac2246fd20a5ed
Sep 10 14:29:27.324225 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.324208    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:29:27.998927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.998891    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" event={"ID":"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b","Type":"ContainerStarted","Data":"d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10"}
Sep 10 14:29:27.999096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.998943    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" event={"ID":"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b","Type":"ContainerStarted","Data":"48236f7f715f5b0d2defbeabff0ae32df2c9395a7664a73791ac2246fd20a5ed"}
Sep 10 14:29:27.999096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:27.999025    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:28.018881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:28.018833    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" podStartSLOduration=1.569923874 podStartE2EDuration="2.018817562s" totalImagesPullingTime="448.893688ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:29:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:28.016903657 +0000 UTC m=+1050.105361231" watchObservedRunningTime="2026-09-10 14:29:28.018817562 +0000 UTC m=+1050.107275133"
Sep 10 14:29:28.427304 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:28.427263    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" path="/var/lib/kubelet/pods/c04d23d2-40a6-4666-bd5e-3dc89cf52ad2/volumes"
Sep 10 14:29:34.009533 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:34.009505    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:29:36.824576 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:36.824541    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:41.011657 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:41.011618    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:29:42.470943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:42.470851    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.535201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:42.535096    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.626632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:42.626584    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:42.727038 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:42.727010    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:43.083223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:43.083185    2576 generic.go:356] "Generic (PLEG): container finished" podID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerID="8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08" exitCode=0
Sep 10 14:29:43.083409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:43.083256    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" event={"ID":"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b","Type":"ContainerDied","Data":"8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"}
Sep 10 14:29:43.083613 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:43.083598    2576 scope.go:117] "RemoveContainer" containerID="8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"
Sep 10 14:29:43.084746 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:43.084715    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:29:43.084882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:43.084785    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:29:43.084882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:43.084817    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:29:44.090317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.090279    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" event={"ID":"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b","Type":"ContainerStarted","Data":"5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d"}
Sep 10 14:29:44.418584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418513    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"]
Sep 10 14:29:44.418894 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418881    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy"
Sep 10 14:29:44.418935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418897    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy"
Sep 10 14:29:44.418935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418907    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy"
Sep 10 14:29:44.418935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418913    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy"
Sep 10 14:29:44.419033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418974    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy"
Sep 10 14:29:44.419033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.418983    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="c04d23d2-40a6-4666-bd5e-3dc89cf52ad2" containerName="istio-proxy"
Sep 10 14:29:44.421148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.421131    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.423582 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.423559    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8f0c5f\"/\"kube-root-ca.crt\""
Sep 10 14:29:44.423896 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.423874    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-8f0c5f\"/\"openshift-service-ca.crt\""
Sep 10 14:29:44.424015 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.423878    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8f0c5f\"/\"test-model-8f0c5f-kserve-self-signed-certs\""
Sep 10 14:29:44.424592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.424574    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-8f0c5f\"/\"default-dockercfg-h8m79\""
Sep 10 14:29:44.431637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.431558    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"]
Sep 10 14:29:44.449656 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.449629    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:44.483173 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.483125    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-home\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.483421 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.483182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z8nw\" (UniqueName: \"kubernetes.io/projected/36cae7e1-0e0b-4dde-a687-2110ff2d772f-kube-api-access-4z8nw\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.483421 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.483263    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-model-cache\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.483558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.483453    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-dshm\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.483558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.483499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tmp-dir\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.483558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.483521    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tls-certs\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584257 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584194    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-model-cache\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584431 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-dshm\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584431 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584366    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tmp-dir\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584431 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584391    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tls-certs\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584578 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584527    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-home\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584578 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584564    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z8nw\" (UniqueName: \"kubernetes.io/projected/36cae7e1-0e0b-4dde-a687-2110ff2d772f-kube-api-access-4z8nw\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584709 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584687    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-model-cache\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584762 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tmp-dir\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.584999 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.584976    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-home\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.586755 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.586738    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-dshm\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.587118 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.587101    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tls-certs\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.593599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.593574    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z8nw\" (UniqueName: \"kubernetes.io/projected/36cae7e1-0e0b-4dde-a687-2110ff2d772f-kube-api-access-4z8nw\") pod \"test-model-8f0c5f-kserve-85974b5c4f-n4qr7\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") " pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:44.621269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.621225    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:44.621477 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.621438    2576 patch_prober.go:28] interesting pod/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 14:29:44.621530 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.621510    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 14:29:44.731826 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:44.731784    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:45.023683 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:45.023651    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"]
Sep 10 14:29:45.025887 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:29:45.025862    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36cae7e1_0e0b_4dde_a687_2110ff2d772f.slice/crio-37228ba0b82f6d4eb5b5a151ae5f2f301250309eae4602b89b4393f4ee558f28 WatchSource:0}: Error finding container 37228ba0b82f6d4eb5b5a151ae5f2f301250309eae4602b89b4393f4ee558f28: Status 404 returned error can't find the container with id 37228ba0b82f6d4eb5b5a151ae5f2f301250309eae4602b89b4393f4ee558f28
Sep 10 14:29:45.097282 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:45.097240    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7" event={"ID":"36cae7e1-0e0b-4dde-a687-2110ff2d772f","Type":"ContainerStarted","Data":"37228ba0b82f6d4eb5b5a151ae5f2f301250309eae4602b89b4393f4ee558f28"}
Sep 10 14:29:45.620671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:45.620641    2576 patch_prober.go:28] interesting pod/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 14:29:45.620842 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:45.620707    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 14:29:46.103736 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:46.103698    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7" event={"ID":"36cae7e1-0e0b-4dde-a687-2110ff2d772f","Type":"ContainerStarted","Data":"b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17"}
Sep 10 14:29:46.118931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:46.118868    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7" podStartSLOduration=2.118852616 podStartE2EDuration="2.118852616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:29:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:29:46.118578033 +0000 UTC m=+1068.207035612" watchObservedRunningTime="2026-09-10 14:29:46.118852616 +0000 UTC m=+1068.207310186"
Sep 10 14:29:46.621281 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:46.621220    2576 patch_prober.go:28] interesting pod/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 14:29:46.621454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:46.621302    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 14:29:47.621033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:47.620998    2576 patch_prober.go:28] interesting pod/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 14:29:47.621451 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:47.621054    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 14:29:48.621555 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:48.621519    2576 patch_prober.go:28] interesting pod/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Sep 10 14:29:48.621941 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:48.621583    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Sep 10 14:29:49.625001 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:49.624922    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:49.625386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:49.625273    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:49.626083 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:49.626067    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:29:49.782097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:49.782062    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:54.732722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:54.732685    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:54.733175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:54.732738    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:54.744956 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:54.744935    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:55.066528 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:55.066441    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:55.159524 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:55.159490    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:29:58.104960 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:58.104924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:29:58.475596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:29:58.475569    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:04.070280 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:04.070224    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:04.358297 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:04.358193    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:04.449797 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:04.449760    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:04.852432 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:04.852386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:07.149562 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:07.149529    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:12.909082 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:12.909041    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:30:13.338459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:13.338427    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:14.724218 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:14.724184    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:15.521521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:15.521479    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:17.778955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:17.778919    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"]
Sep 10 14:30:17.779465 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:17.779180    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" podUID="63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" containerName="maas-api" containerID="cri-o://d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10" gracePeriod=30
Sep 10 14:30:17.807130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:17.807103    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:17.907747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:17.907709    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:17.980634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:17.980603    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:18.032577 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.032506    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:30:18.078905 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.078881    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls\") pod \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") "
Sep 10 14:30:18.079073 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.078912    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-metrics-certs\") pod \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") "
Sep 10 14:30:18.079073 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.079010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-openshift-service-ca\") pod \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") "
Sep 10 14:30:18.079073 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.079051    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjxdt\" (UniqueName: \"kubernetes.io/projected/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-kube-api-access-wjxdt\") pod \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\" (UID: \"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b\") "
Sep 10 14:30:18.079405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.079383    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" (UID: "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:30:18.081420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.081389    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" (UID: "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:30:18.081420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.081404    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-kube-api-access-wjxdt" (OuterVolumeSpecName: "kube-api-access-wjxdt") pod "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" (UID: "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b"). InnerVolumeSpecName "kube-api-access-wjxdt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:30:18.081664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.081639    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" (UID: "63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:30:18.180364 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.180331    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:18.180364 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.180357    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjxdt\" (UniqueName: \"kubernetes.io/projected/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-kube-api-access-wjxdt\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:18.180364 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.180369    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:18.180585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.180378    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:18.270144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.270105    2576 generic.go:356] "Generic (PLEG): container finished" podID="63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" containerID="d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10" exitCode=0
Sep 10 14:30:18.270359 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.270170    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"
Sep 10 14:30:18.270359 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.270181    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" event={"ID":"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b","Type":"ContainerDied","Data":"d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10"}
Sep 10 14:30:18.270359 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.270252    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2" event={"ID":"63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b","Type":"ContainerDied","Data":"48236f7f715f5b0d2defbeabff0ae32df2c9395a7664a73791ac2246fd20a5ed"}
Sep 10 14:30:18.270359 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.270278    2576 scope.go:117] "RemoveContainer" containerID="d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10"
Sep 10 14:30:18.279354 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.279336    2576 scope.go:117] "RemoveContainer" containerID="d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10"
Sep 10 14:30:18.279656 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:30:18.279632    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10\": container with ID starting with d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10 not found: ID does not exist" containerID="d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10"
Sep 10 14:30:18.279722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.279670    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10"} err="failed to get container status \"d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10\": rpc error: code = NotFound desc = could not find container \"d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10\": container with ID starting with d3a0a5cccbda600d1356e5697f3bd37f0a57c66ab4576c84abdf76be8a52ca10 not found: ID does not exist"
Sep 10 14:30:18.289730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.289676    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"]
Sep 10 14:30:18.292269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.292248    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:18.292352 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.292316    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-ba183c-85b99794d9-xl6l2"]
Sep 10 14:30:18.427863 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:18.427828    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" path="/var/lib/kubelet/pods/63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b/volumes"
Sep 10 14:30:21.253673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:21.253634    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:21.321671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:21.321638    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:21.404403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:21.404371    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:21.505327 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:21.505253    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:22.127369 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:22.127334    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:22.228427 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:22.228394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:22.326569 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:22.326540    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:23.368771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:23.368719    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:23.710589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:23.710561    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:25.204389 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:25.204350    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:30:29.191133 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:29.191075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:29.412372 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:29.412338    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"]
Sep 10 14:30:29.412677 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:29.412624    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" containerID="cri-o://5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d" gracePeriod=30
Sep 10 14:30:29.431071 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:29.431039    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:29.841979 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:29.841945    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:30.258780 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:30.258740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:30.417459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:30.417430    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:30.517938 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:30.517863    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:31.064288 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:31.064255    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:31.823160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:31.823123    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:31.925955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:31.925926    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:34.669710 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.669686    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:30:34.732439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732351    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istiod-ca-cert\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732405    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-socket\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732458    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-podinfo\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732491    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-envoy\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q92l5\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-kube-api-access-q92l5\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732607    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-data\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732670    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-token\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732716    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-certs\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732752    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-credential-socket\") pod \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\" (UID: \"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b\") "
Sep 10 14:30:34.732883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732762    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:34.732883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732817    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:30:34.733145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.732903    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-data" (OuterVolumeSpecName: "istio-data") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:34.733145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.733050    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.733145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.733069    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.733145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.733077    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:34.733145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.733082    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.733145 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.733100    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:34.735076 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.735046    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-token" (OuterVolumeSpecName: "istio-token") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:30:34.735252 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.735205    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-kube-api-access-q92l5" (OuterVolumeSpecName: "kube-api-access-q92l5") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "kube-api-access-q92l5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:30:34.735538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.735521    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:30:34.735590 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.735546    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" (UID: "52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:30:34.833768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.833734    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.833768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.833761    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.833768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.833770    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.833768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.833778    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.834039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.833786    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q92l5\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-kube-api-access-q92l5\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:34.834039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:34.833794    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:30:35.357344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.357306    2576 generic.go:356] "Generic (PLEG): container finished" podID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerID="5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d" exitCode=0
Sep 10 14:30:35.357524 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.357351    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" event={"ID":"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b","Type":"ContainerDied","Data":"5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d"}
Sep 10 14:30:35.357524 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.357374    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"
Sep 10 14:30:35.357524 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.357411    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" event={"ID":"52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b","Type":"ContainerDied","Data":"36782c251b95860a4cec52d6893bd6a9d41de7633db32cc6ba61bc5c721d7a1f"}
Sep 10 14:30:35.357524 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.357434    2576 scope.go:117] "RemoveContainer" containerID="5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d"
Sep 10 14:30:35.366840 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.366820    2576 scope.go:117] "RemoveContainer" containerID="8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"
Sep 10 14:30:35.374875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.374858    2576 scope.go:117] "RemoveContainer" containerID="5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d"
Sep 10 14:30:35.375146 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:30:35.375125    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d\": container with ID starting with 5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d not found: ID does not exist" containerID="5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d"
Sep 10 14:30:35.375224 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.375153    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d"} err="failed to get container status \"5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d\": rpc error: code = NotFound desc = could not find container \"5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d\": container with ID starting with 5ec8669860ee9a71fd7d0aa198476a0f4851198c9b177ec3e04d692b93cf893d not found: ID does not exist"
Sep 10 14:30:35.375224 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.375170    2576 scope.go:117] "RemoveContainer" containerID="8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"
Sep 10 14:30:35.375432 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:30:35.375416    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08\": container with ID starting with 8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08 not found: ID does not exist" containerID="8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"
Sep 10 14:30:35.375478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.375440    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08"} err="failed to get container status \"8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08\": rpc error: code = NotFound desc = could not find container \"8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08\": container with ID starting with 8b3b55709ae80696b54f0e468b5ea8f1e5ede9fadff47f1cc870525b59d3fd08 not found: ID does not exist"
Sep 10 14:30:35.380160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.380138    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"]
Sep 10 14:30:35.385616 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.385591    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97"]
Sep 10 14:30:35.626706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.626601    2576 patch_prober.go:28] interesting pod/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 14:30:35.626706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:35.626688    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-ba183c-openshift-default-7d9fd68d64-5tv97" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 14:30:36.427714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:36.427681    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" path="/var/lib/kubelet/pods/52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b/volumes"
Sep 10 14:30:36.803090 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:36.803009    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:36.878005 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:36.877973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:37.038586 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.038547    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:37.308409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.308365    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"]
Sep 10 14:30:37.308814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.308796    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy"
Sep 10 14:30:37.308909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.308818    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy"
Sep 10 14:30:37.313254 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.309385    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" containerName="maas-api"
Sep 10 14:30:37.313254 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.309411    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" containerName="maas-api"
Sep 10 14:30:37.317495 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.317440    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy"
Sep 10 14:30:37.317720 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.317484    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="63b8a2a3-7d96-4f8c-b4f9-7dee0a94b11b" containerName="maas-api"
Sep 10 14:30:37.317720 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.317528    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy"
Sep 10 14:30:37.321207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.321189    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.323870 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.323854    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:30:37.323998 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.323980    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7c910967-openshift-default-dockercfg-bb2ss\""
Sep 10 14:30:37.329172 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.329146    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"]
Sep 10 14:30:37.344070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.344037    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:37.456497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456462    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-certs\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456502    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-549fd\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-kube-api-access-549fd\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456557    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-istio-token\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-socket\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456605    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c0437-4b32-4546-a930-2f13d7688102-istio-podinfo\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456737    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-envoy\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456785    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c0437-4b32-4546-a930-2f13d7688102-istiod-ca-cert\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456835    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-data\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.456953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.456867    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-credential-socket\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.476551 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.476521    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:37.558126 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558089    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c0437-4b32-4546-a930-2f13d7688102-istio-podinfo\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558150    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-envoy\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558170    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c0437-4b32-4546-a930-2f13d7688102-istiod-ca-cert\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-data\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558225    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-credential-socket\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-certs\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558315    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-549fd\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-kube-api-access-549fd\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558360    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-istio-token\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558389    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-socket\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.558682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558660    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-data\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.559004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558974    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-socket\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.559004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.558992    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c0437-4b32-4546-a930-2f13d7688102-istiod-ca-cert\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.559169 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.559043    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-credential-socket\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.559169 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.559112    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-certs\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.561342 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.561317    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-envoy\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.561500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.561483    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c0437-4b32-4546-a930-2f13d7688102-istio-podinfo\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.566751 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.566729    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-istio-token\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.566963 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.566941    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-549fd\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-kube-api-access-549fd\") pod \"e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") " pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.634682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.634647    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:37.910379 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.910202    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"]
Sep 10 14:30:37.913210 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:30:37.913185    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod685c0437_4b32_4546_a930_2f13d7688102.slice/crio-3038038b406b273bf619699fb695c0d2c22d0d1518e1b360c6017be53259657f WatchSource:0}: Error finding container 3038038b406b273bf619699fb695c0d2c22d0d1518e1b360c6017be53259657f: Status 404 returned error can't find the container with id 3038038b406b273bf619699fb695c0d2c22d0d1518e1b360c6017be53259657f
Sep 10 14:30:37.915616 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.915575    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:30:37.915707 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.915662    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:30:37.915747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:37.915707    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:30:38.376013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:38.375978    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" event={"ID":"685c0437-4b32-4546-a930-2f13d7688102","Type":"ContainerStarted","Data":"d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"}
Sep 10 14:30:38.376413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:38.376039    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" event={"ID":"685c0437-4b32-4546-a930-2f13d7688102","Type":"ContainerStarted","Data":"3038038b406b273bf619699fb695c0d2c22d0d1518e1b360c6017be53259657f"}
Sep 10 14:30:38.396979 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:38.396914    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podStartSLOduration=1.396899194 podStartE2EDuration="1.396899194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:30:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:30:38.394793316 +0000 UTC m=+1120.483250887" watchObservedRunningTime="2026-09-10 14:30:38.396899194 +0000 UTC m=+1120.485356807"
Sep 10 14:30:38.635330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:38.635257    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:38.636952 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:38.636928    2576 patch_prober.go:28] interesting pod/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 10 14:30:38.637057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:38.636971    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 10 14:30:39.639854 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:39.639826    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:40.270092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:40.270056    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:40.373086 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:40.373051    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:40.385368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:40.385334    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:40.386360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:40.386340    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:30:41.308174 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.308144    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:30:41.424604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.424572    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"]
Sep 10 14:30:41.425023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.425007    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy"
Sep 10 14:30:41.425070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.425026    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="52dacbfe-8c9f-4fe5-8f84-a0c3eec4de9b" containerName="istio-proxy"
Sep 10 14:30:41.427079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.427064    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.429664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.429642    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-7c910967-serving-cert\""
Sep 10 14:30:41.440463 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.440434    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"]
Sep 10 14:30:41.498203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.498159    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.498484 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.498248    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-metrics-certs\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.498484 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.498403    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05321450-b9da-42a0-93fc-2ec1a0d188ae-openshift-service-ca\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.498696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.498670    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xstx\" (UniqueName: \"kubernetes.io/projected/05321450-b9da-42a0-93fc-2ec1a0d188ae-kube-api-access-2xstx\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.521122 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.521091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:41.599772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.599674    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xstx\" (UniqueName: \"kubernetes.io/projected/05321450-b9da-42a0-93fc-2ec1a0d188ae-kube-api-access-2xstx\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.599942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.599769    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.599942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.599809    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-metrics-certs\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.599942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.599858    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05321450-b9da-42a0-93fc-2ec1a0d188ae-openshift-service-ca\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.599942 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:30:41.599930    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-serving-cert: secret "maas-api-e2e-mt-7c910967-serving-cert" not found
Sep 10 14:30:41.600167 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:30:41.600007    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls podName:05321450-b9da-42a0-93fc-2ec1a0d188ae nodeName:}" failed. No retries permitted until 2026-09-10 14:30:42.09998712 +0000 UTC m=+1124.188444674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls") pod "maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" (UID: "05321450-b9da-42a0-93fc-2ec1a0d188ae") : secret "maas-api-e2e-mt-7c910967-serving-cert" not found
Sep 10 14:30:41.600558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.600537    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05321450-b9da-42a0-93fc-2ec1a0d188ae-openshift-service-ca\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.603102 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.603074    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-metrics-certs\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.609636 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.609613    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xstx\" (UniqueName: \"kubernetes.io/projected/05321450-b9da-42a0-93fc-2ec1a0d188ae-kube-api-access-2xstx\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:41.622341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.622316    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:41.722421 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.722381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:41.953592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:41.953562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:42.044464 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:42.044434    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:42.105915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:42.105884    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:42.108536 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:42.108478    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls\") pod \"maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:42.338653 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:42.338566    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:42.610701 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:42.610669    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"]
Sep 10 14:30:42.613155 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:30:42.613129    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05321450_b9da_42a0_93fc_2ec1a0d188ae.slice/crio-2e1500bbc4fb30daee9fdaadf9c39f19e23319b8548c343626593c3d363ba34f WatchSource:0}: Error finding container 2e1500bbc4fb30daee9fdaadf9c39f19e23319b8548c343626593c3d363ba34f: Status 404 returned error can't find the container with id 2e1500bbc4fb30daee9fdaadf9c39f19e23319b8548c343626593c3d363ba34f
Sep 10 14:30:43.402004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.401921    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" event={"ID":"05321450-b9da-42a0-93fc-2ec1a0d188ae","Type":"ContainerStarted","Data":"45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2"}
Sep 10 14:30:43.402004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.401984    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" event={"ID":"05321450-b9da-42a0-93fc-2ec1a0d188ae","Type":"ContainerStarted","Data":"2e1500bbc4fb30daee9fdaadf9c39f19e23319b8548c343626593c3d363ba34f"}
Sep 10 14:30:43.402432 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.402042    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:43.419641 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.419590    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" podStartSLOduration=1.979934205 podStartE2EDuration="2.419574171s" totalImagesPullingTime="439.639966ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:30:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:30:43.418451954 +0000 UTC m=+1125.506909528" watchObservedRunningTime="2026-09-10 14:30:43.419574171 +0000 UTC m=+1125.508031819"
Sep 10 14:30:43.891214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.891170    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"]
Sep 10 14:30:43.893869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.893849    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:43.897422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.897392    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-215950\"/\"default-dockercfg-dt2ts\""
Sep 10 14:30:43.897422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.897415    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-215950\"/\"openshift-service-ca.crt\""
Sep 10 14:30:43.897633 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.897446    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-noten-215950\"/\"kube-root-ca.crt\""
Sep 10 14:30:43.897633 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.897458    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-noten-215950\"/\"noten-model-215950-kserve-self-signed-certs\""
Sep 10 14:30:43.902661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:43.902639    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"]
Sep 10 14:30:44.022172 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.022132    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401110c2-c67a-4228-883c-5bfac9201c6c-tls-certs\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.022172 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.022174    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbqbd\" (UniqueName: \"kubernetes.io/projected/401110c2-c67a-4228-883c-5bfac9201c6c-kube-api-access-fbqbd\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.022420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.022208    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-tmp-dir\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.022420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.022299    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-model-cache\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.022420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.022333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-dshm\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.022420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.022363    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-home\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.102464 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.102432    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:44.122848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.122820    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401110c2-c67a-4228-883c-5bfac9201c6c-tls-certs\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.122848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.122852    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbqbd\" (UniqueName: \"kubernetes.io/projected/401110c2-c67a-4228-883c-5bfac9201c6c-kube-api-access-fbqbd\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.122874    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-tmp-dir\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.122927    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-model-cache\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.122961    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-dshm\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.122997    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-home\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.123385    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-tmp-dir\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.123399    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-home\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.123501 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.123451    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-model-cache\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.125390 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.125369    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-dshm\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.125764 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.125745    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401110c2-c67a-4228-883c-5bfac9201c6c-tls-certs\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.130476 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.130454    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbqbd\" (UniqueName: \"kubernetes.io/projected/401110c2-c67a-4228-883c-5bfac9201c6c-kube-api-access-fbqbd\") pod \"noten-model-215950-kserve-84b595fc4c-82w7q\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") " pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.203602 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.203559    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:44.480720 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:44.480695    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"]
Sep 10 14:30:44.483086 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:30:44.483059    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod401110c2_c67a_4228_883c_5bfac9201c6c.slice/crio-ef75d2d77c2662ad6f23e09c4385636a2ac01ac4283d7bfee34107344ea93b1e WatchSource:0}: Error finding container ef75d2d77c2662ad6f23e09c4385636a2ac01ac4283d7bfee34107344ea93b1e: Status 404 returned error can't find the container with id ef75d2d77c2662ad6f23e09c4385636a2ac01ac4283d7bfee34107344ea93b1e
Sep 10 14:30:45.413774 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:45.413733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q" event={"ID":"401110c2-c67a-4228-883c-5bfac9201c6c","Type":"ContainerStarted","Data":"b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a"}
Sep 10 14:30:45.413990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:45.413782    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q" event={"ID":"401110c2-c67a-4228-883c-5bfac9201c6c","Type":"ContainerStarted","Data":"ef75d2d77c2662ad6f23e09c4385636a2ac01ac4283d7bfee34107344ea93b1e"}
Sep 10 14:30:45.430314 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:45.430267    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q" podStartSLOduration=2.430251814 podStartE2EDuration="2.430251814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:30:45.429169772 +0000 UTC m=+1127.517627334" watchObservedRunningTime="2026-09-10 14:30:45.430251814 +0000 UTC m=+1127.518709730"
Sep 10 14:30:49.412130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:49.412096    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:30:54.204321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:54.204275    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:54.204321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:54.204330    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:54.217002 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:54.216978    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:54.488534 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:54.488449    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:30:56.825772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:56.825735    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:30:57.730491 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:57.730455    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"]
Sep 10 14:30:57.890420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:57.890378    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:57.942596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:57.942567    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:58.088341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:58.088257    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:58.196651 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:58.196604    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:59.492747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:59.492711    2576 generic.go:356] "Generic (PLEG): container finished" podID="685c0437-4b32-4546-a930-2f13d7688102" containerID="d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24" exitCode=0
Sep 10 14:30:59.493169 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:59.492778    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" event={"ID":"685c0437-4b32-4546-a930-2f13d7688102","Type":"ContainerDied","Data":"d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"}
Sep 10 14:30:59.493169 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:59.493121    2576 scope.go:117] "RemoveContainer" containerID="d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"
Sep 10 14:30:59.494151 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:59.494124    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:30:59.494209 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:59.494191    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:30:59.494264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:30:59.494225    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:00.500077 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:00.500045    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" event={"ID":"685c0437-4b32-4546-a930-2f13d7688102","Type":"ContainerStarted","Data":"14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083"}
Sep 10 14:31:00.634942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:00.634909    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:31:00.635176 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:00.635154    2576 patch_prober.go:28] interesting pod/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 10 14:31:00.635337 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:00.635213    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 10 14:31:01.634974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:01.634942    2576 patch_prober.go:28] interesting pod/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 10 14:31:01.635436 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:01.634996    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 10 14:31:01.635964 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:01.635941    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:02.635945 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:02.635908    2576 patch_prober.go:28] interesting pod/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 10 14:31:02.636366 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:02.635971    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 10 14:31:03.038683 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:03.038649    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"]
Sep 10 14:31:03.068092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:03.068030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:03.224923 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:03.224894    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:03.345720 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:03.345639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:03.635376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:03.635283    2576 patch_prober.go:28] interesting pod/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 10 14:31:03.635376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:03.635342    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 10 14:31:04.024496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:04.024457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:04.635943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:04.635903    2576 patch_prober.go:28] interesting pod/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body=
Sep 10 14:31:04.636139 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:04.635964    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused"
Sep 10 14:31:05.203846 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:05.203808    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.640088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:05.639987    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:31:05.640539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:05.640485    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:31:05.641274 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:05.641255    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:31:05.836974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:05.836943    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:05.932925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:05.932893    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:06.254554 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:06.254470    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:12.013797 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:12.013762    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:31:12.441812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:12.441778    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:12.541719 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:12.541683    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:12.753566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:12.753465    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q" podUID="401110c2-c67a-4228-883c-5bfac9201c6c" containerName="main" containerID="cri-o://b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a" gracePeriod=45
Sep 10 14:31:13.008918 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.008862    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:31:13.096650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096614    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-home\") pod \"401110c2-c67a-4228-883c-5bfac9201c6c\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") "
Sep 10 14:31:13.096650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096647    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-dshm\") pod \"401110c2-c67a-4228-883c-5bfac9201c6c\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") "
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096723    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401110c2-c67a-4228-883c-5bfac9201c6c-tls-certs\") pod \"401110c2-c67a-4228-883c-5bfac9201c6c\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") "
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096750    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-model-cache\") pod \"401110c2-c67a-4228-883c-5bfac9201c6c\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") "
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbqbd\" (UniqueName: \"kubernetes.io/projected/401110c2-c67a-4228-883c-5bfac9201c6c-kube-api-access-fbqbd\") pod \"401110c2-c67a-4228-883c-5bfac9201c6c\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") "
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096803    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-tmp-dir\") pod \"401110c2-c67a-4228-883c-5bfac9201c6c\" (UID: \"401110c2-c67a-4228-883c-5bfac9201c6c\") "
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.096905    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-home" (OuterVolumeSpecName: "home") pod "401110c2-c67a-4228-883c-5bfac9201c6c" (UID: "401110c2-c67a-4228-883c-5bfac9201c6c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.097032    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-model-cache" (OuterVolumeSpecName: "model-cache") pod "401110c2-c67a-4228-883c-5bfac9201c6c" (UID: "401110c2-c67a-4228-883c-5bfac9201c6c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.097111    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "401110c2-c67a-4228-883c-5bfac9201c6c" (UID: "401110c2-c67a-4228-883c-5bfac9201c6c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:13.097162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.097164    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:13.097523 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.097177    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:13.099175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.099154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-dshm" (OuterVolumeSpecName: "dshm") pod "401110c2-c67a-4228-883c-5bfac9201c6c" (UID: "401110c2-c67a-4228-883c-5bfac9201c6c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:13.099284 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.099256    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/401110c2-c67a-4228-883c-5bfac9201c6c-kube-api-access-fbqbd" (OuterVolumeSpecName: "kube-api-access-fbqbd") pod "401110c2-c67a-4228-883c-5bfac9201c6c" (UID: "401110c2-c67a-4228-883c-5bfac9201c6c"). InnerVolumeSpecName "kube-api-access-fbqbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:13.099326 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.099266    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/401110c2-c67a-4228-883c-5bfac9201c6c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "401110c2-c67a-4228-883c-5bfac9201c6c" (UID: "401110c2-c67a-4228-883c-5bfac9201c6c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:13.197741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.197702    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:13.197741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.197735    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401110c2-c67a-4228-883c-5bfac9201c6c-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:13.197940 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.197752    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbqbd\" (UniqueName: \"kubernetes.io/projected/401110c2-c67a-4228-883c-5bfac9201c6c-kube-api-access-fbqbd\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:13.197940 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.197765    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401110c2-c67a-4228-883c-5bfac9201c6c-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:13.531689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.531647    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:13.570119 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.570085    2576 generic.go:356] "Generic (PLEG): container finished" podID="401110c2-c67a-4228-883c-5bfac9201c6c" containerID="b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a" exitCode=0
Sep 10 14:31:13.570317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.570131    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"
Sep 10 14:31:13.570317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.570174    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q" event={"ID":"401110c2-c67a-4228-883c-5bfac9201c6c","Type":"ContainerDied","Data":"b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a"}
Sep 10 14:31:13.570317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.570219    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q" event={"ID":"401110c2-c67a-4228-883c-5bfac9201c6c","Type":"ContainerDied","Data":"ef75d2d77c2662ad6f23e09c4385636a2ac01ac4283d7bfee34107344ea93b1e"}
Sep 10 14:31:13.570317 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.570267    2576 scope.go:117] "RemoveContainer" containerID="b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a"
Sep 10 14:31:13.579299 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.579209    2576 scope.go:117] "RemoveContainer" containerID="b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a"
Sep 10 14:31:13.585532 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:13.579940    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a\": container with ID starting with b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a not found: ID does not exist" containerID="b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a"
Sep 10 14:31:13.585532 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.579989    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a"} err="failed to get container status \"b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a\": rpc error: code = NotFound desc = could not find container \"b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a\": container with ID starting with b4f5b98a891731145c14a9039509ec0be1985ab27a21609a58ec477f1cd4257a not found: ID does not exist"
Sep 10 14:31:13.592631 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.592606    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"]
Sep 10 14:31:13.594137 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.594116    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-noten-215950/noten-model-215950-kserve-84b595fc4c-82w7q"]
Sep 10 14:31:13.648163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.648127    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:13.713746 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.713712    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:13.778667 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.778635    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:13.837965 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:13.837885    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:14.132732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:14.132649    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:14.427955 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:14.427920    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="401110c2-c67a-4228-883c-5bfac9201c6c" path="/var/lib/kubelet/pods/401110c2-c67a-4228-883c-5bfac9201c6c/volumes"
Sep 10 14:31:18.087129 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.087092    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7" podUID="36cae7e1-0e0b-4dde-a687-2110ff2d772f" containerName="main" containerID="cri-o://b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17" gracePeriod=45
Sep 10 14:31:18.334487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.334463    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:31:18.444409 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444385    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tls-certs\") pod \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") "
Sep 10 14:31:18.444585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444421    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tmp-dir\") pod \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") "
Sep 10 14:31:18.444585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444444    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-home\") pod \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") "
Sep 10 14:31:18.444585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444468    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-model-cache\") pod \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") "
Sep 10 14:31:18.444585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444516    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-dshm\") pod \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") "
Sep 10 14:31:18.444585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444560    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4z8nw\" (UniqueName: \"kubernetes.io/projected/36cae7e1-0e0b-4dde-a687-2110ff2d772f-kube-api-access-4z8nw\") pod \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\" (UID: \"36cae7e1-0e0b-4dde-a687-2110ff2d772f\") "
Sep 10 14:31:18.444848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444712    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36cae7e1-0e0b-4dde-a687-2110ff2d772f" (UID: "36cae7e1-0e0b-4dde-a687-2110ff2d772f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:18.444848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444730    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-home" (OuterVolumeSpecName: "home") pod "36cae7e1-0e0b-4dde-a687-2110ff2d772f" (UID: "36cae7e1-0e0b-4dde-a687-2110ff2d772f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:18.444848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444742    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-model-cache" (OuterVolumeSpecName: "model-cache") pod "36cae7e1-0e0b-4dde-a687-2110ff2d772f" (UID: "36cae7e1-0e0b-4dde-a687-2110ff2d772f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:18.444848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444811    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:18.444848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444827    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:18.444848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.444838    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:18.446893 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.446865    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36cae7e1-0e0b-4dde-a687-2110ff2d772f" (UID: "36cae7e1-0e0b-4dde-a687-2110ff2d772f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:18.446994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.446900    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36cae7e1-0e0b-4dde-a687-2110ff2d772f-kube-api-access-4z8nw" (OuterVolumeSpecName: "kube-api-access-4z8nw") pod "36cae7e1-0e0b-4dde-a687-2110ff2d772f" (UID: "36cae7e1-0e0b-4dde-a687-2110ff2d772f"). InnerVolumeSpecName "kube-api-access-4z8nw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:18.446994 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.446931    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-dshm" (OuterVolumeSpecName: "dshm") pod "36cae7e1-0e0b-4dde-a687-2110ff2d772f" (UID: "36cae7e1-0e0b-4dde-a687-2110ff2d772f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:18.545538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.545497    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4z8nw\" (UniqueName: \"kubernetes.io/projected/36cae7e1-0e0b-4dde-a687-2110ff2d772f-kube-api-access-4z8nw\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:18.545664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.545548    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36cae7e1-0e0b-4dde-a687-2110ff2d772f-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:18.545664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.545558    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36cae7e1-0e0b-4dde-a687-2110ff2d772f-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:18.597371 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.597337    2576 generic.go:356] "Generic (PLEG): container finished" podID="36cae7e1-0e0b-4dde-a687-2110ff2d772f" containerID="b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17" exitCode=0
Sep 10 14:31:18.597540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.597373    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7" event={"ID":"36cae7e1-0e0b-4dde-a687-2110ff2d772f","Type":"ContainerDied","Data":"b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17"}
Sep 10 14:31:18.597540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.597389    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"
Sep 10 14:31:18.597540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.597425    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7" event={"ID":"36cae7e1-0e0b-4dde-a687-2110ff2d772f","Type":"ContainerDied","Data":"37228ba0b82f6d4eb5b5a151ae5f2f301250309eae4602b89b4393f4ee558f28"}
Sep 10 14:31:18.597540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.597449    2576 scope.go:117] "RemoveContainer" containerID="b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17"
Sep 10 14:31:18.607828 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.607811    2576 scope.go:117] "RemoveContainer" containerID="b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17"
Sep 10 14:31:18.608106 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:18.608087    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17\": container with ID starting with b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17 not found: ID does not exist" containerID="b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17"
Sep 10 14:31:18.608174 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.608117    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17"} err="failed to get container status \"b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17\": rpc error: code = NotFound desc = could not find container \"b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17\": container with ID starting with b4698c8eebc452d28153f2b64395339bf19c01b63644093df129a4035a40fb17 not found: ID does not exist"
Sep 10 14:31:18.620581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.620548    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"]
Sep 10 14:31:18.624061 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:18.624039    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-8f0c5f/test-model-8f0c5f-kserve-85974b5c4f-n4qr7"]
Sep 10 14:31:20.431661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:20.431592    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36cae7e1-0e0b-4dde-a687-2110ff2d772f" path="/var/lib/kubelet/pods/36cae7e1-0e0b-4dde-a687-2110ff2d772f/volumes"
Sep 10 14:31:20.512555 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:20.512527    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:20.646606 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:20.646570    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:20.803778 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:20.803687    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:21.343826 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:21.343776    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:21.718371 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:21.718343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:31:23.070397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.070366    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:23.595800 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.595763    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"]
Sep 10 14:31:23.596024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.595998    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" podUID="05321450-b9da-42a0-93fc-2ec1a0d188ae" containerName="maas-api" containerID="cri-o://45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2" gracePeriod=30
Sep 10 14:31:23.625223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.625197    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:23.725785 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.725759    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:23.813348 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.813308    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:23.843405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.843384    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:31:23.882078 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.882007    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:23.893901 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.893877    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-metrics-certs\") pod \"05321450-b9da-42a0-93fc-2ec1a0d188ae\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") "
Sep 10 14:31:23.894024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.893957    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls\") pod \"05321450-b9da-42a0-93fc-2ec1a0d188ae\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") "
Sep 10 14:31:23.894024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.893986    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xstx\" (UniqueName: \"kubernetes.io/projected/05321450-b9da-42a0-93fc-2ec1a0d188ae-kube-api-access-2xstx\") pod \"05321450-b9da-42a0-93fc-2ec1a0d188ae\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") "
Sep 10 14:31:23.894159 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.894077    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05321450-b9da-42a0-93fc-2ec1a0d188ae-openshift-service-ca\") pod \"05321450-b9da-42a0-93fc-2ec1a0d188ae\" (UID: \"05321450-b9da-42a0-93fc-2ec1a0d188ae\") "
Sep 10 14:31:23.894492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.894469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05321450-b9da-42a0-93fc-2ec1a0d188ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "05321450-b9da-42a0-93fc-2ec1a0d188ae" (UID: "05321450-b9da-42a0-93fc-2ec1a0d188ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:23.896260 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.896205    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05321450-b9da-42a0-93fc-2ec1a0d188ae-kube-api-access-2xstx" (OuterVolumeSpecName: "kube-api-access-2xstx") pod "05321450-b9da-42a0-93fc-2ec1a0d188ae" (UID: "05321450-b9da-42a0-93fc-2ec1a0d188ae"). InnerVolumeSpecName "kube-api-access-2xstx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:23.896674 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.896647    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "05321450-b9da-42a0-93fc-2ec1a0d188ae" (UID: "05321450-b9da-42a0-93fc-2ec1a0d188ae"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:23.896674 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.896665    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "05321450-b9da-42a0-93fc-2ec1a0d188ae" (UID: "05321450-b9da-42a0-93fc-2ec1a0d188ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:23.995661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.995628    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:23.995661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.995655    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xstx\" (UniqueName: \"kubernetes.io/projected/05321450-b9da-42a0-93fc-2ec1a0d188ae-kube-api-access-2xstx\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:23.995661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.995665    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/05321450-b9da-42a0-93fc-2ec1a0d188ae-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:23.995661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:23.995673    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/05321450-b9da-42a0-93fc-2ec1a0d188ae-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:24.160864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.160781    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:24.627439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.627408    2576 generic.go:356] "Generic (PLEG): container finished" podID="05321450-b9da-42a0-93fc-2ec1a0d188ae" containerID="45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2" exitCode=0
Sep 10 14:31:24.627612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.627456    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"
Sep 10 14:31:24.627612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.627499    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" event={"ID":"05321450-b9da-42a0-93fc-2ec1a0d188ae","Type":"ContainerDied","Data":"45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2"}
Sep 10 14:31:24.627612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.627550    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl" event={"ID":"05321450-b9da-42a0-93fc-2ec1a0d188ae","Type":"ContainerDied","Data":"2e1500bbc4fb30daee9fdaadf9c39f19e23319b8548c343626593c3d363ba34f"}
Sep 10 14:31:24.627612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.627574    2576 scope.go:117] "RemoveContainer" containerID="45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2"
Sep 10 14:31:24.638379 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.638361    2576 scope.go:117] "RemoveContainer" containerID="45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2"
Sep 10 14:31:24.638699 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:24.638676    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2\": container with ID starting with 45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2 not found: ID does not exist" containerID="45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2"
Sep 10 14:31:24.638753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.638699    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2"} err="failed to get container status \"45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2\": rpc error: code = NotFound desc = could not find container \"45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2\": container with ID starting with 45daa43f83adbfdb2ea96056c8a6845439fea3d9d46d89b8653db5b2813be2d2 not found: ID does not exist"
Sep 10 14:31:24.646670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.646640    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"]
Sep 10 14:31:24.656330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:24.656304    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-7c910967-6d798bb9cd-b9tdl"]
Sep 10 14:31:26.427245 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:26.427203    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05321450-b9da-42a0-93fc-2ec1a0d188ae" path="/var/lib/kubelet/pods/05321450-b9da-42a0-93fc-2ec1a0d188ae/volumes"
Sep 10 14:31:30.133613 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.133581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:30.211663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.211630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:30.377424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.377392    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"]
Sep 10 14:31:30.377655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.377625    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy" containerID="cri-o://14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083" gracePeriod=30
Sep 10 14:31:30.404840 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.404763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:30.737740 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.737707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:30.825503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.825473    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:30.939544 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:30.939517    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:31.254007 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:31.253974    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:31.823540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:31.823498    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"]
Sep 10 14:31:31.823779 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:31.823750    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" podUID="11da1529-b807-4403-a07d-7121e47d8959" containerName="maas-api" containerID="cri-o://788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21" gracePeriod=30
Sep 10 14:31:31.859434 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:31.859409    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:31.960373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:31.960335    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:32.059750 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.059692    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:32.066255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.066222    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:31:32.128979 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.128905    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:32.171120 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.171091    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fx8sc\" (UniqueName: \"kubernetes.io/projected/11da1529-b807-4403-a07d-7121e47d8959-kube-api-access-fx8sc\") pod \"11da1529-b807-4403-a07d-7121e47d8959\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") "
Sep 10 14:31:32.171309 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.171141    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11da1529-b807-4403-a07d-7121e47d8959-openshift-service-ca\") pod \"11da1529-b807-4403-a07d-7121e47d8959\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") "
Sep 10 14:31:32.171309 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.171222    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls\") pod \"11da1529-b807-4403-a07d-7121e47d8959\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") "
Sep 10 14:31:32.171309 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.171262    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-metrics-certs\") pod \"11da1529-b807-4403-a07d-7121e47d8959\" (UID: \"11da1529-b807-4403-a07d-7121e47d8959\") "
Sep 10 14:31:32.171582 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.171558    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11da1529-b807-4403-a07d-7121e47d8959-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "11da1529-b807-4403-a07d-7121e47d8959" (UID: "11da1529-b807-4403-a07d-7121e47d8959"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:32.173466 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.173436    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11da1529-b807-4403-a07d-7121e47d8959-kube-api-access-fx8sc" (OuterVolumeSpecName: "kube-api-access-fx8sc") pod "11da1529-b807-4403-a07d-7121e47d8959" (UID: "11da1529-b807-4403-a07d-7121e47d8959"). InnerVolumeSpecName "kube-api-access-fx8sc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:32.173622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.173444    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "11da1529-b807-4403-a07d-7121e47d8959" (UID: "11da1529-b807-4403-a07d-7121e47d8959"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:32.173622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.173476    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "11da1529-b807-4403-a07d-7121e47d8959" (UID: "11da1529-b807-4403-a07d-7121e47d8959"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:32.272420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.272385    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fx8sc\" (UniqueName: \"kubernetes.io/projected/11da1529-b807-4403-a07d-7121e47d8959-kube-api-access-fx8sc\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.272420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.272414    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/11da1529-b807-4403-a07d-7121e47d8959-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.272420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.272426    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.272835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.272436    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/11da1529-b807-4403-a07d-7121e47d8959-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:32.392219 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.392135    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:32.512373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.512335    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:31:32.668639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.668541    2576 generic.go:356] "Generic (PLEG): container finished" podID="11da1529-b807-4403-a07d-7121e47d8959" containerID="788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21" exitCode=0
Sep 10 14:31:32.668639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.668601    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"
Sep 10 14:31:32.668639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.668609    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" event={"ID":"11da1529-b807-4403-a07d-7121e47d8959","Type":"ContainerDied","Data":"788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21"}
Sep 10 14:31:32.668886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.668665    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9" event={"ID":"11da1529-b807-4403-a07d-7121e47d8959","Type":"ContainerDied","Data":"b28290173bee89b5e579e826785010b8608f39ec357a0fa0effe5d431cd716ec"}
Sep 10 14:31:32.668886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.668707    2576 scope.go:117] "RemoveContainer" containerID="788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21"
Sep 10 14:31:32.677619 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.677597    2576 scope.go:117] "RemoveContainer" containerID="788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21"
Sep 10 14:31:32.677925 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:32.677897    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21\": container with ID starting with 788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21 not found: ID does not exist" containerID="788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21"
Sep 10 14:31:32.678026 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.677928    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21"} err="failed to get container status \"788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21\": rpc error: code = NotFound desc = could not find container \"788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21\": container with ID starting with 788cc8c265437d1436572ff243fecdc5f5d9661f34ccd6edbe1ce0118ce66a21 not found: ID does not exist"
Sep 10 14:31:32.686384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.686360    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"]
Sep 10 14:31:32.690175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:32.690152    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-8f0c5f-5c9dc8d98c-58cd9"]
Sep 10 14:31:34.427016 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:34.426986    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11da1529-b807-4403-a07d-7121e47d8959" path="/var/lib/kubelet/pods/11da1529-b807-4403-a07d-7121e47d8959/volumes"
Sep 10 14:31:35.634063 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.634040    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:31:35.686958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.686926    2576 generic.go:356] "Generic (PLEG): container finished" podID="685c0437-4b32-4546-a930-2f13d7688102" containerID="14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083" exitCode=0
Sep 10 14:31:35.687133 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.686985    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" event={"ID":"685c0437-4b32-4546-a930-2f13d7688102","Type":"ContainerDied","Data":"14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083"}
Sep 10 14:31:35.687133 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.686995    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"
Sep 10 14:31:35.687133 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.687028    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp" event={"ID":"685c0437-4b32-4546-a930-2f13d7688102","Type":"ContainerDied","Data":"3038038b406b273bf619699fb695c0d2c22d0d1518e1b360c6017be53259657f"}
Sep 10 14:31:35.687133 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.687051    2576 scope.go:117] "RemoveContainer" containerID="14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083"
Sep 10 14:31:35.696458 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.696436    2576 scope.go:117] "RemoveContainer" containerID="d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"
Sep 10 14:31:35.703858 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.703839    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-certs\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.703982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.703894    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-549fd\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-kube-api-access-549fd\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.703982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.703919    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-credential-socket\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.703982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.703944    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-data\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.703982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.703959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-envoy\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.704201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.703987    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c0437-4b32-4546-a930-2f13d7688102-istio-podinfo\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.704201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704024    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-istio-token\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.704201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704066    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-socket\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.704201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704108    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c0437-4b32-4546-a930-2f13d7688102-istiod-ca-cert\") pod \"685c0437-4b32-4546-a930-2f13d7688102\" (UID: \"685c0437-4b32-4546-a930-2f13d7688102\") "
Sep 10 14:31:35.704438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704223    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:35.704438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704394    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.704438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704414    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-data" (OuterVolumeSpecName: "istio-data") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:35.704438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704427    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:35.704652 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704597    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:35.704652 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704627    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/685c0437-4b32-4546-a930-2f13d7688102-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:35.705108 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.704981    2576 scope.go:117] "RemoveContainer" containerID="14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083"
Sep 10 14:31:35.705597 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:35.705569    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083\": container with ID starting with 14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083 not found: ID does not exist" containerID="14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083"
Sep 10 14:31:35.705715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.705608    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083"} err="failed to get container status \"14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083\": rpc error: code = NotFound desc = could not find container \"14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083\": container with ID starting with 14c2b37ec5d1f5b0198054d45c80b9b5e8f63a7f645e9d3705d275da7e0bc083 not found: ID does not exist"
Sep 10 14:31:35.705715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.705633    2576 scope.go:117] "RemoveContainer" containerID="d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"
Sep 10 14:31:35.706009 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:35.705984    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24\": container with ID starting with d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24 not found: ID does not exist" containerID="d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"
Sep 10 14:31:35.706122 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.706015    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24"} err="failed to get container status \"d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24\": rpc error: code = NotFound desc = could not find container \"d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24\": container with ID starting with d1a0d44f1b7b7dac20ab47c78761d30f2c991885fc39f0558b2fbb22bd25ff24 not found: ID does not exist"
Sep 10 14:31:35.706560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.706533    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-kube-api-access-549fd" (OuterVolumeSpecName: "kube-api-access-549fd") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "kube-api-access-549fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:35.706778 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.706751    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/685c0437-4b32-4546-a930-2f13d7688102-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:31:35.706859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.706804    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-istio-token" (OuterVolumeSpecName: "istio-token") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:35.706899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.706848    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "685c0437-4b32-4546-a930-2f13d7688102" (UID: "685c0437-4b32-4546-a930-2f13d7688102"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:31:35.804794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804758    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.804794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804787    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.804794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804797    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.804794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804806    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/685c0437-4b32-4546-a930-2f13d7688102-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.805056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804813    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.805056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804820    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/685c0437-4b32-4546-a930-2f13d7688102-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.805056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804828    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/685c0437-4b32-4546-a930-2f13d7688102-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:35.805056 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:35.804836    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-549fd\" (UniqueName: \"kubernetes.io/projected/685c0437-4b32-4546-a930-2f13d7688102-kube-api-access-549fd\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:31:36.009450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:36.009422    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"]
Sep 10 14:31:36.012736 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:36.012715    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7c910967-openshift-default-7fdc59fbc6-c8pkp"]
Sep 10 14:31:36.427263 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:36.427217    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="685c0437-4b32-4546-a930-2f13d7688102" path="/var/lib/kubelet/pods/685c0437-4b32-4546-a930-2f13d7688102/volumes"
Sep 10 14:31:37.402587 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:37.402537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:37.518540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:37.518506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:37.694381 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:37.694346    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:37.882562 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:37.882526    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:39.129454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:39.129425    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:39.228476 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:39.228448    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:39.303461 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:39.303429    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:39.723426 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:39.723390    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:31:39.817157 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:39.817077    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:40.089549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089469    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"]
Sep 10 14:31:40.089818 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089805    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089819    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089828    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11da1529-b807-4403-a07d-7121e47d8959" containerName="maas-api"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089833    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="11da1529-b807-4403-a07d-7121e47d8959" containerName="maas-api"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089842    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05321450-b9da-42a0-93fc-2ec1a0d188ae" containerName="maas-api"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089848    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="05321450-b9da-42a0-93fc-2ec1a0d188ae" containerName="maas-api"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089853    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="401110c2-c67a-4228-883c-5bfac9201c6c" containerName="main"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089858    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="401110c2-c67a-4228-883c-5bfac9201c6c" containerName="main"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089868    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36cae7e1-0e0b-4dde-a687-2110ff2d772f" containerName="main"
Sep 10 14:31:40.089875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089873    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="36cae7e1-0e0b-4dde-a687-2110ff2d772f" containerName="main"
Sep 10 14:31:40.090195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089940    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="401110c2-c67a-4228-883c-5bfac9201c6c" containerName="main"
Sep 10 14:31:40.090195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089947    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="05321450-b9da-42a0-93fc-2ec1a0d188ae" containerName="maas-api"
Sep 10 14:31:40.090195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089953    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy"
Sep 10 14:31:40.090195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089959    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy"
Sep 10 14:31:40.090195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089964    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="36cae7e1-0e0b-4dde-a687-2110ff2d772f" containerName="main"
Sep 10 14:31:40.090195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.089969    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="11da1529-b807-4403-a07d-7121e47d8959" containerName="maas-api"
Sep 10 14:31:40.093696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.093676    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.096211 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.096191    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-586826-openshift-default-dockercfg-sf7fn\""
Sep 10 14:31:40.096332 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.096194    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:31:40.108334 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.108311    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"]
Sep 10 14:31:40.203003 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.202971    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:40.241646 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241621    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8efc1563-83e0-4080-abd9-f219a5231e01-istio-podinfo\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241662    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8efc1563-83e0-4080-abd9-f219a5231e01-istiod-ca-cert\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241690    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-socket\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241712    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm794\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-kube-api-access-xm794\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241774    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-data\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241822    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-envoy\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241854    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-istio-token\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241889    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-credential-socket\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.241932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.241916    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-certs\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342498    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-data\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342539    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-envoy\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342560    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-istio-token\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342585    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-credential-socket\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342612    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-certs\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342700    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8efc1563-83e0-4080-abd9-f219a5231e01-istio-podinfo\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342740    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8efc1563-83e0-4080-abd9-f219a5231e01-istiod-ca-cert\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342766    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-socket\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.342881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342787    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm794\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-kube-api-access-xm794\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.343255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.342967    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-data\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.343255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.343046    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-credential-socket\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.343255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.343199    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-certs\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.343366 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.343255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-socket\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.343736 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.343713    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8efc1563-83e0-4080-abd9-f219a5231e01-istiod-ca-cert\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.345634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.345612    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-envoy\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.345950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.345925    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8efc1563-83e0-4080-abd9-f219a5231e01-istio-podinfo\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.352204 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.352175    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm794\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-kube-api-access-xm794\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.352345 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.352277    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-istio-token\") pod \"e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") " pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.406079 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.406049    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:40.467581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.467490    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:40.689318 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.689199    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"]
Sep 10 14:31:40.692045 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:31:40.692019    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8efc1563_83e0_4080_abd9_f219a5231e01.slice/crio-547eca6a3217037cb41b9ae5fc2011999f40447002ba5564ed67eea175181000 WatchSource:0}: Error finding container 547eca6a3217037cb41b9ae5fc2011999f40447002ba5564ed67eea175181000: Status 404 returned error can't find the container with id 547eca6a3217037cb41b9ae5fc2011999f40447002ba5564ed67eea175181000
Sep 10 14:31:40.694337 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.694304    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:40.694408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.694370    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:40.694455 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.694413    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:40.713771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.713739    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" event={"ID":"8efc1563-83e0-4080-abd9-f219a5231e01","Type":"ContainerStarted","Data":"547eca6a3217037cb41b9ae5fc2011999f40447002ba5564ed67eea175181000"}
Sep 10 14:31:40.766316 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:40.766285    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:41.720746 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:41.720711    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" event={"ID":"8efc1563-83e0-4080-abd9-f219a5231e01","Type":"ContainerStarted","Data":"e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"}
Sep 10 14:31:41.740843 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:41.740791    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podStartSLOduration=1.740773766 podStartE2EDuration="1.740773766s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:31:41.739265392 +0000 UTC m=+1183.827722954" watchObservedRunningTime="2026-09-10 14:31:41.740773766 +0000 UTC m=+1183.829231337"
Sep 10 14:31:42.406859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:42.406820    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:42.411788 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:42.411760    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:42.726714 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:42.726687    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:42.727786 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:42.727769    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:31:43.624786 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:43.624754    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:43.719594 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:43.719556    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:44.564633 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:44.564596    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:44.877940 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:44.877848    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:44.973028 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:44.972978    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:45.333118 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:45.333081    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:45.420377 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:45.420339    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:46.333352 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:46.333324    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:47.794384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:47.794350    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:47.888415 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:47.888379    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:48.210968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.210931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:31:48.914743 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.914708    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"]
Sep 10 14:31:48.915116 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.915082    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy"
Sep 10 14:31:48.915116 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.915093    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="685c0437-4b32-4546-a930-2f13d7688102" containerName="istio-proxy"
Sep 10 14:31:48.918575 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.918555    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:48.921142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.921122    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:31:48.921505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.921487    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:31:48.922055 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.922039    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-9e5db6-serving-cert\""
Sep 10 14:31:48.922151 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.922079    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:31:48.922217 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.922186    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:31:48.923391 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.923367    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551fd97-8eeb-45e7-b213-764d24674072-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:48.923504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.923404    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zxvv\" (UniqueName: \"kubernetes.io/projected/b551fd97-8eeb-45e7-b213-764d24674072-kube-api-access-2zxvv\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:48.923504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.923491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:48.923578 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.923537    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:48.928954 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:48.928931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"]
Sep 10 14:31:49.024463 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.024433    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.024632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.024488    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.024632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.024551    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551fd97-8eeb-45e7-b213-764d24674072-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.024632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.024581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zxvv\" (UniqueName: \"kubernetes.io/projected/b551fd97-8eeb-45e7-b213-764d24674072-kube-api-access-2zxvv\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.024758 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:49.024651    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-serving-cert: secret "maas-api-e2e-shared-a-w2-9e5db6-serving-cert" not found
Sep 10 14:31:49.024758 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:31:49.024732    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls podName:b551fd97-8eeb-45e7-b213-764d24674072 nodeName:}" failed. No retries permitted until 2026-09-10 14:31:49.524710442 +0000 UTC m=+1191.613168005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls") pod "maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" (UID: "b551fd97-8eeb-45e7-b213-764d24674072") : secret "maas-api-e2e-shared-a-w2-9e5db6-serving-cert" not found
Sep 10 14:31:49.025326 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.025259    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551fd97-8eeb-45e7-b213-764d24674072-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.026979 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.026960    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.034176 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.034137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.035636 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.035616    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zxvv\" (UniqueName: \"kubernetes.io/projected/b551fd97-8eeb-45e7-b213-764d24674072-kube-api-access-2zxvv\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.130957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.130927    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.423809 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.423774    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.530467 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.530437    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.532984 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.532958    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:49.570656 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.570626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.671166 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.671137    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:49.830350 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:49.830262    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:50.103876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:50.103435    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"]
Sep 10 14:31:50.775246 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:50.775170    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" event={"ID":"b551fd97-8eeb-45e7-b213-764d24674072","Type":"ContainerStarted","Data":"e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84"}
Sep 10 14:31:50.775450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:50.775278    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" event={"ID":"b551fd97-8eeb-45e7-b213-764d24674072","Type":"ContainerStarted","Data":"3a5c39dc2a87dbe528196eed4bc7fd713f6e25510ce94386957429f84ab3c8e8"}
Sep 10 14:31:50.776165 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:50.776135    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:50.797087 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:50.796990    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" podStartSLOduration=2.372736512 podStartE2EDuration="2.796951505s" totalImagesPullingTime="424.214993ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:31:50.794278839 +0000 UTC m=+1192.882736407" watchObservedRunningTime="2026-09-10 14:31:50.796951505 +0000 UTC m=+1192.885409075"
Sep 10 14:31:57.793639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:57.793605    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:31:58.386091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.386052    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:31:58.404574 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.403165    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:31:58.487948 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.487919    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": read tcp 10.134.0.2:33878->10.134.0.51:15021: read: connection reset by peer" start-of-body=
Sep 10 14:31:58.488066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.487984    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": read tcp 10.134.0.2:33878->10.134.0.51:15021: read: connection reset by peer"
Sep 10 14:31:58.818186 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.818154    2576 generic.go:356] "Generic (PLEG): container finished" podID="8efc1563-83e0-4080-abd9-f219a5231e01" containerID="e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698" exitCode=0
Sep 10 14:31:58.818612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.818259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" event={"ID":"8efc1563-83e0-4080-abd9-f219a5231e01","Type":"ContainerDied","Data":"e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"}
Sep 10 14:31:58.818679 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.818665    2576 scope.go:117] "RemoveContainer" containerID="e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"
Sep 10 14:31:58.819767 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.819741    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:58.819828 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.819807    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:58.819870 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:58.819840    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:31:59.825517 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:31:59.825483    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" event={"ID":"8efc1563-83e0-4080-abd9-f219a5231e01","Type":"ContainerStarted","Data":"9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30"}
Sep 10 14:32:00.129107 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:00.129028    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:32:00.406702 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:00.406606    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:32:00.406875 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:00.406853    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 14:32:00.406932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:00.406913    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 14:32:01.406957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:01.406919    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 14:32:01.407396 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:01.406997    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 14:32:02.407038 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:02.406995    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 14:32:02.407495 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:02.407061    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 14:32:03.407370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:03.407341    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 14:32:03.407823 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:03.407799    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 14:32:04.407362 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:04.407324    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Sep 10 14:32:04.407546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:04.407379    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Sep 10 14:32:05.425216 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:05.425186    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:32:05.425716 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:05.425537    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:32:05.426537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:05.426511    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:32:05.812884 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:05.812784    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:08.561062 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:08.561021    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:08.885284 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:08.885181    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:09.151779 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.151693    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"]
Sep 10 14:32:09.155566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.155544    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.158080 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.158058    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-9dcc39-openshift-default-dockercfg-zzk9g\""
Sep 10 14:32:09.168218 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.168190    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"]
Sep 10 14:32:09.181794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.181770    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:09.221065 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221037    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-token\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221096    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50f2c681-ce81-4278-926a-d08b2f5a2929-istiod-ca-cert\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221116    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-data\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221170    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-socket\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221207    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn6jb\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-kube-api-access-jn6jb\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-certs\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221279    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-credential-socket\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221300    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-podinfo\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.221499 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.221333    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-envoy\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.319757 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.319728    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:09.321930 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.321905    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50f2c681-ce81-4278-926a-d08b2f5a2929-istiod-ca-cert\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.321944    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-data\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.321979    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-socket\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322021    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn6jb\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-kube-api-access-jn6jb\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322051    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-certs\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322082    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-credential-socket\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322112    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-podinfo\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322142    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-envoy\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322190    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-token\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322477    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-certs\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322573 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322550    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-socket\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322573 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322566    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-data\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322675 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322651    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-credential-socket\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.322747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.322708    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50f2c681-ce81-4278-926a-d08b2f5a2929-istiod-ca-cert\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.324762 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.324737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-podinfo\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.324871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.324814    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-envoy\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.330549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.330523    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn6jb\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-kube-api-access-jn6jb\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.330639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.330549    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-token\") pod \"e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") " pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.469418 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.469380    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:09.746135 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.746074    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"]
Sep 10 14:32:09.748341 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:32:09.748315    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50f2c681_ce81_4278_926a_d08b2f5a2929.slice/crio-419ca124018818a64f90514f7a4cf8e1b7495360e4e3442ae24d318c4355f6e5 WatchSource:0}: Error finding container 419ca124018818a64f90514f7a4cf8e1b7495360e4e3442ae24d318c4355f6e5: Status 404 returned error can't find the container with id 419ca124018818a64f90514f7a4cf8e1b7495360e4e3442ae24d318c4355f6e5
Sep 10 14:32:09.750656 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.750610    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:09.750751 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.750703    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:09.750791 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.750752    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:09.884212 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.884176    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" event={"ID":"50f2c681-ce81-4278-926a-d08b2f5a2929","Type":"ContainerStarted","Data":"da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"}
Sep 10 14:32:09.884480 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.884260    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" event={"ID":"50f2c681-ce81-4278-926a-d08b2f5a2929","Type":"ContainerStarted","Data":"419ca124018818a64f90514f7a4cf8e1b7495360e4e3442ae24d318c4355f6e5"}
Sep 10 14:32:09.920752 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.920690    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podStartSLOduration=0.920671697 podStartE2EDuration="920.671697ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:09.919734367 +0000 UTC m=+1212.008191939" watchObservedRunningTime="2026-09-10 14:32:09.920671697 +0000 UTC m=+1212.009129268"
Sep 10 14:32:09.990014 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:09.989986    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:10.470311 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:10.470279    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:10.474918 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:10.474898    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:10.888849 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:10.888765    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:10.889919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:10.889900    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:11.820762 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:11.820724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:11.918802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:11.918769    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:12.752033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:12.752002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.081201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:13.081114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.182849 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:13.182814    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.366328 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:13.366221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.638422 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:13.638329    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.771382 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:13.771351    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:13.859962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:13.859928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:16.106907 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:16.106872    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:16.208439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:16.208401    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.428223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.428191    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"]
Sep 10 14:32:17.431255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.431203    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.433761 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.433736    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-54c040-serving-cert\""
Sep 10 14:32:17.447175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.447149    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"]
Sep 10 14:32:17.472479 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.472453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.497478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.497450    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh2bt\" (UniqueName: \"kubernetes.io/projected/7f16b67f-6fed-42bc-a888-41f6f786ef1f-kube-api-access-nh2bt\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.497593 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.497499    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.497659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.497615    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16b67f-6fed-42bc-a888-41f6f786ef1f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.497756 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.497739    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.571493 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.571454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.598933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.598900    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.599096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.598957    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16b67f-6fed-42bc-a888-41f6f786ef1f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.599096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.599014    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.599096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.599052    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh2bt\" (UniqueName: \"kubernetes.io/projected/7f16b67f-6fed-42bc-a888-41f6f786ef1f-kube-api-access-nh2bt\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.599740 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.599717    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16b67f-6fed-42bc-a888-41f6f786ef1f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.601536 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.601517    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.601642 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.601596    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.610395 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.610374    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh2bt\" (UniqueName: \"kubernetes.io/projected/7f16b67f-6fed-42bc-a888-41f6f786ef1f-kube-api-access-nh2bt\") pod \"maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.671761 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.671738    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.742223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.742141    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:17.935903 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:17.935868    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:18.027497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.027395    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"]
Sep 10 14:32:18.034191 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:32:18.034159    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f16b67f_6fed_42bc_a888_41f6f786ef1f.slice/crio-5a923bb731c0cd76d62f02a94cc627f08055738e98697505969335712f42fb54 WatchSource:0}: Error finding container 5a923bb731c0cd76d62f02a94cc627f08055738e98697505969335712f42fb54: Status 404 returned error can't find the container with id 5a923bb731c0cd76d62f02a94cc627f08055738e98697505969335712f42fb54
Sep 10 14:32:18.049161 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.049128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:18.431272 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.431224    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:32:18.927913 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.927882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9" event={"ID":"7f16b67f-6fed-42bc-a888-41f6f786ef1f","Type":"ContainerStarted","Data":"0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829"}
Sep 10 14:32:18.928114 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.927932    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9" event={"ID":"7f16b67f-6fed-42bc-a888-41f6f786ef1f","Type":"ContainerStarted","Data":"5a923bb731c0cd76d62f02a94cc627f08055738e98697505969335712f42fb54"}
Sep 10 14:32:18.928114 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.927985    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:18.949223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:18.949176    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9" podStartSLOduration=1.402692028 podStartE2EDuration="1.949159801s" totalImagesPullingTime="546.467773ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:18.946768896 +0000 UTC m=+1221.035226466" watchObservedRunningTime="2026-09-10 14:32:18.949159801 +0000 UTC m=+1221.037617372"
Sep 10 14:32:24.938318 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:24.938290    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:32:26.930437 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:26.930394    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 10 14:32:26.930900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:26.930471    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 10 14:32:28.986196 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:28.986167    2576 generic.go:356] "Generic (PLEG): container finished" podID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerID="da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9" exitCode=0
Sep 10 14:32:28.986690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:28.986259    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" event={"ID":"50f2c681-ce81-4278-926a-d08b2f5a2929","Type":"ContainerDied","Data":"da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"}
Sep 10 14:32:28.986781 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:28.986763    2576 scope.go:117] "RemoveContainer" containerID="da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"
Sep 10 14:32:28.987925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:28.987897    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:28.988015 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:28.987961    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:28.988015 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:28.988006    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:29.993376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:29.993346    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" event={"ID":"50f2c681-ce81-4278-926a-d08b2f5a2929","Type":"ContainerStarted","Data":"9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889"}
Sep 10 14:32:30.471006 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:30.470948    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:32:30.471266 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:30.471031    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:32:30.471416 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:30.471395    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:31.319798 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:31.319758    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:32:31.470370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:31.470322    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:32:31.470589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:31.470394    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:32:32.470735 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:32.470695    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:32:32.471348 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:32.471315    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:32:33.470020 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:33.469974    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:32:33.470248 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:33.470052    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:32:34.105062 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:34.105030    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:34.469769 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:34.469735    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 10 14:32:34.470073 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:34.470045    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 10 14:32:35.476158 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:35.476119    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:35.476638 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:35.476365    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:35.477120 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:35.477095    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:32:36.753035 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:36.752992    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:39.395701 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:39.395668    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:41.274601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:41.274562    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:41.814637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:41.814601    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7596-ce6dcc"]
Sep 10 14:32:41.818009 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:41.817990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:41.820637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:41.820619    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2hkkb\""
Sep 10 14:32:41.824094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:41.824069    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-ce6dcc"]
Sep 10 14:32:41.922779 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:41.922745    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zwpbg\" (UniqueName: \"kubernetes.io/projected/ae7cd018-03c7-4843-a9a7-41fc57469c72-kube-api-access-zwpbg\") pod \"test-curl-7596-ce6dcc\" (UID: \"ae7cd018-03c7-4843-a9a7-41fc57469c72\") " pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:42.023622 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:42.023581    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zwpbg\" (UniqueName: \"kubernetes.io/projected/ae7cd018-03c7-4843-a9a7-41fc57469c72-kube-api-access-zwpbg\") pod \"test-curl-7596-ce6dcc\" (UID: \"ae7cd018-03c7-4843-a9a7-41fc57469c72\") " pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:42.031767 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:42.031737    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zwpbg\" (UniqueName: \"kubernetes.io/projected/ae7cd018-03c7-4843-a9a7-41fc57469c72-kube-api-access-zwpbg\") pod \"test-curl-7596-ce6dcc\" (UID: \"ae7cd018-03c7-4843-a9a7-41fc57469c72\") " pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:42.128975 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:42.128885    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:42.397028 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:42.397001    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-ce6dcc"]
Sep 10 14:32:42.398960 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:32:42.398933    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podae7cd018_03c7_4843_a9a7_41fc57469c72.slice/crio-6f2871a86536afff443eb2c52990597fdaf6d86655717d50cf4ea301380b1f12 WatchSource:0}: Error finding container 6f2871a86536afff443eb2c52990597fdaf6d86655717d50cf4ea301380b1f12: Status 404 returned error can't find the container with id 6f2871a86536afff443eb2c52990597fdaf6d86655717d50cf4ea301380b1f12
Sep 10 14:32:43.058873 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.058836    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-ce6dcc" event={"ID":"ae7cd018-03c7-4843-a9a7-41fc57469c72","Type":"ContainerStarted","Data":"0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f"}
Sep 10 14:32:43.059053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.058883    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-ce6dcc" event={"ID":"ae7cd018-03c7-4843-a9a7-41fc57469c72","Type":"ContainerStarted","Data":"6f2871a86536afff443eb2c52990597fdaf6d86655717d50cf4ea301380b1f12"}
Sep 10 14:32:43.073411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.073185    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7596-ce6dcc" podStartSLOduration=2.073167967 podStartE2EDuration="2.073167967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:43.07289013 +0000 UTC m=+1245.161347701" watchObservedRunningTime="2026-09-10 14:32:43.073167967 +0000 UTC m=+1245.161625540"
Sep 10 14:32:43.644163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.644126    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7596-ce6dcc"]
Sep 10 14:32:43.647895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.647867    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7596-ce6dcc"]
Sep 10 14:32:43.765729 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.765644    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7596-89a411"]
Sep 10 14:32:43.766066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.766052    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" containerName="test-curl-7596-ce6dcc"
Sep 10 14:32:43.766109 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.766069    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" containerName="test-curl-7596-ce6dcc"
Sep 10 14:32:43.766159 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.766149    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" containerName="test-curl-7596-ce6dcc"
Sep 10 14:32:43.768335 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.768318    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:43.773675 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.773649    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-89a411"]
Sep 10 14:32:43.838608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.838561    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc9ww\" (UniqueName: \"kubernetes.io/projected/44e9151a-79e3-4e56-b7ef-6af98b137290-kube-api-access-mc9ww\") pod \"test-curl-7596-89a411\" (UID: \"44e9151a-79e3-4e56-b7ef-6af98b137290\") " pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:43.939646 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.939611    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc9ww\" (UniqueName: \"kubernetes.io/projected/44e9151a-79e3-4e56-b7ef-6af98b137290-kube-api-access-mc9ww\") pod \"test-curl-7596-89a411\" (UID: \"44e9151a-79e3-4e56-b7ef-6af98b137290\") " pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:43.947741 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:43.947722    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc9ww\" (UniqueName: \"kubernetes.io/projected/44e9151a-79e3-4e56-b7ef-6af98b137290-kube-api-access-mc9ww\") pod \"test-curl-7596-89a411\" (UID: \"44e9151a-79e3-4e56-b7ef-6af98b137290\") " pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:44.064278 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:44.064172    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7596-ce6dcc" container="test-curl-7596-ce6dcc"
Sep 10 14:32:44.079059 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:44.079038    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:44.352650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:44.352625    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-89a411"]
Sep 10 14:32:44.354947 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:32:44.354909    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod44e9151a_79e3_4e56_b7ef_6af98b137290.slice/crio-2589385a121051da2181c364fcaea3ea9165ffa37f73e9797c5364d72ab18f1f WatchSource:0}: Error finding container 2589385a121051da2181c364fcaea3ea9165ffa37f73e9797c5364d72ab18f1f: Status 404 returned error can't find the container with id 2589385a121051da2181c364fcaea3ea9165ffa37f73e9797c5364d72ab18f1f
Sep 10 14:32:45.070790 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.070754    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-89a411" event={"ID":"44e9151a-79e3-4e56-b7ef-6af98b137290","Type":"ContainerStarted","Data":"433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2"}
Sep 10 14:32:45.070790 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.070796    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-89a411" event={"ID":"44e9151a-79e3-4e56-b7ef-6af98b137290","Type":"ContainerStarted","Data":"2589385a121051da2181c364fcaea3ea9165ffa37f73e9797c5364d72ab18f1f"}
Sep 10 14:32:45.071443 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.070829    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7596-ce6dcc" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" containerName="test-curl-7596-ce6dcc" containerID="cri-o://0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f" gracePeriod=2
Sep 10 14:32:45.717222 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.717155    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7596-89a411" podStartSLOduration=2.717133167 podStartE2EDuration="2.717133167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:45.084412116 +0000 UTC m=+1247.172869687" watchObservedRunningTime="2026-09-10 14:32:45.717133167 +0000 UTC m=+1247.805590738"
Sep 10 14:32:45.718160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.718134    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7596-89a411"]
Sep 10 14:32:45.722961 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.722939    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7596-89a411"]
Sep 10 14:32:45.838015 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.837926    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7596-d014ac"]
Sep 10 14:32:45.838453 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.838433    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44e9151a-79e3-4e56-b7ef-6af98b137290" containerName="test-curl-7596-89a411"
Sep 10 14:32:45.838564 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.838455    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="44e9151a-79e3-4e56-b7ef-6af98b137290" containerName="test-curl-7596-89a411"
Sep 10 14:32:45.838564 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.838548    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="44e9151a-79e3-4e56-b7ef-6af98b137290" containerName="test-curl-7596-89a411"
Sep 10 14:32:45.840818 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.840799    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:45.847267 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.847226    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-d014ac"]
Sep 10 14:32:45.857442 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.857415    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8lm7\" (UniqueName: \"kubernetes.io/projected/38f2b246-b970-46f3-a173-1f248707c846-kube-api-access-t8lm7\") pod \"test-curl-7596-d014ac\" (UID: \"38f2b246-b970-46f3-a173-1f248707c846\") " pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:45.958039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.958001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8lm7\" (UniqueName: \"kubernetes.io/projected/38f2b246-b970-46f3-a173-1f248707c846-kube-api-access-t8lm7\") pod \"test-curl-7596-d014ac\" (UID: \"38f2b246-b970-46f3-a173-1f248707c846\") " pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:45.969707 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:45.969681    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8lm7\" (UniqueName: \"kubernetes.io/projected/38f2b246-b970-46f3-a173-1f248707c846-kube-api-access-t8lm7\") pod \"test-curl-7596-d014ac\" (UID: \"38f2b246-b970-46f3-a173-1f248707c846\") " pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:46.076065 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:46.076034    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7596-89a411" container="test-curl-7596-89a411"
Sep 10 14:32:46.151341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:46.151225    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:46.423974 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:32:46.423944    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38f2b246_b970_46f3_a173_1f248707c846.slice/crio-44a7cd5407b65c440d66103e6fe248cb27ac29cf2eabdb476f6c287c9d9b363f WatchSource:0}: Error finding container 44a7cd5407b65c440d66103e6fe248cb27ac29cf2eabdb476f6c287c9d9b363f: Status 404 returned error can't find the container with id 44a7cd5407b65c440d66103e6fe248cb27ac29cf2eabdb476f6c287c9d9b363f
Sep 10 14:32:46.429127 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:46.429104    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-d014ac"]
Sep 10 14:32:47.083267 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.083208    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-d014ac" event={"ID":"38f2b246-b970-46f3-a173-1f248707c846","Type":"ContainerStarted","Data":"ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600"}
Sep 10 14:32:47.083627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.083275    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-d014ac" event={"ID":"38f2b246-b970-46f3-a173-1f248707c846","Type":"ContainerStarted","Data":"44a7cd5407b65c440d66103e6fe248cb27ac29cf2eabdb476f6c287c9d9b363f"}
Sep 10 14:32:47.083627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.083288    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7596-89a411" podUID="44e9151a-79e3-4e56-b7ef-6af98b137290" containerName="test-curl-7596-89a411" containerID="cri-o://433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2" gracePeriod=2
Sep 10 14:32:47.100552 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.100510    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7596-d014ac" podStartSLOduration=2.100494436 podStartE2EDuration="2.100494436s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:47.098514788 +0000 UTC m=+1249.186972362" watchObservedRunningTime="2026-09-10 14:32:47.100494436 +0000 UTC m=+1249.188952007"
Sep 10 14:32:47.257734 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.257712    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:47.372998 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.372914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zwpbg\" (UniqueName: \"kubernetes.io/projected/ae7cd018-03c7-4843-a9a7-41fc57469c72-kube-api-access-zwpbg\") pod \"ae7cd018-03c7-4843-a9a7-41fc57469c72\" (UID: \"ae7cd018-03c7-4843-a9a7-41fc57469c72\") "
Sep 10 14:32:47.375006 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.374980    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae7cd018-03c7-4843-a9a7-41fc57469c72-kube-api-access-zwpbg" (OuterVolumeSpecName: "kube-api-access-zwpbg") pod "ae7cd018-03c7-4843-a9a7-41fc57469c72" (UID: "ae7cd018-03c7-4843-a9a7-41fc57469c72"). InnerVolumeSpecName "kube-api-access-zwpbg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:47.474307 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.474282    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zwpbg\" (UniqueName: \"kubernetes.io/projected/ae7cd018-03c7-4843-a9a7-41fc57469c72-kube-api-access-zwpbg\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:32:47.667643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.667553    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7596-d014ac"]
Sep 10 14:32:47.671302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.671276    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7596-d014ac"]
Sep 10 14:32:47.788424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.788391    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7596-b3cb7d"]
Sep 10 14:32:47.788825 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.788811    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38f2b246-b970-46f3-a173-1f248707c846" containerName="test-curl-7596-d014ac"
Sep 10 14:32:47.788888 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.788828    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="38f2b246-b970-46f3-a173-1f248707c846" containerName="test-curl-7596-d014ac"
Sep 10 14:32:47.788942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.788904    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="38f2b246-b970-46f3-a173-1f248707c846" containerName="test-curl-7596-d014ac"
Sep 10 14:32:47.791247 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.791198    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:47.798408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.798363    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-b3cb7d"]
Sep 10 14:32:47.878330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.878296    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjjcm\" (UniqueName: \"kubernetes.io/projected/0418e94d-7da0-41b8-9ab8-cd45df8d04fb-kube-api-access-zjjcm\") pod \"test-curl-7596-b3cb7d\" (UID: \"0418e94d-7da0-41b8-9ab8-cd45df8d04fb\") " pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:47.979045 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.979011    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjjcm\" (UniqueName: \"kubernetes.io/projected/0418e94d-7da0-41b8-9ab8-cd45df8d04fb-kube-api-access-zjjcm\") pod \"test-curl-7596-b3cb7d\" (UID: \"0418e94d-7da0-41b8-9ab8-cd45df8d04fb\") " pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:47.988687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:47.988664    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjjcm\" (UniqueName: \"kubernetes.io/projected/0418e94d-7da0-41b8-9ab8-cd45df8d04fb-kube-api-access-zjjcm\") pod \"test-curl-7596-b3cb7d\" (UID: \"0418e94d-7da0-41b8-9ab8-cd45df8d04fb\") " pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:48.089575 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.089533    2576 generic.go:356] "Generic (PLEG): container finished" podID="ae7cd018-03c7-4843-a9a7-41fc57469c72" containerID="0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f" exitCode=137
Sep 10 14:32:48.090023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.089584    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:48.090023 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.089631    2576 scope.go:117] "RemoveContainer" containerID="0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f"
Sep 10 14:32:48.090023 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:48.089958    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7596-d014ac" container="test-curl-7596-d014ac"
Sep 10 14:32:48.099021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.099005    2576 scope.go:117] "RemoveContainer" containerID="0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f"
Sep 10 14:32:48.099319 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:48.099300    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f\": container with ID starting with 0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f not found: ID does not exist" containerID="0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f"
Sep 10 14:32:48.099378 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.099326    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f"} err="failed to get container status \"0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f\": rpc error: code = NotFound desc = could not find container \"0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f\": container with ID starting with 0fadc432d188f8148e8f248acbcc6e09c2d9b2003321b5f7d55a5f945652a66f not found: ID does not exist"
Sep 10 14:32:48.100941 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:48.100921    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7596-ce6dcc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:48.101197 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.101184    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:48.371620 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.371562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7596-b3cb7d"]
Sep 10 14:32:48.374159 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:32:48.374136    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0418e94d_7da0_41b8_9ab8_cd45df8d04fb.slice/crio-78faa3bbf0c5584120710eebb27f39d6a12ce622a0c0a7aa5306f069b6b7fa0a WatchSource:0}: Error finding container 78faa3bbf0c5584120710eebb27f39d6a12ce622a0c0a7aa5306f069b6b7fa0a: Status 404 returned error can't find the container with id 78faa3bbf0c5584120710eebb27f39d6a12ce622a0c0a7aa5306f069b6b7fa0a
Sep 10 14:32:48.426555 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:48.426522    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7596-ce6dcc\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" pod="opendatahub/test-curl-7596-ce6dcc"
Sep 10 14:32:48.428445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:48.428421    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae7cd018-03c7-4843-a9a7-41fc57469c72" path="/var/lib/kubelet/pods/ae7cd018-03c7-4843-a9a7-41fc57469c72/volumes"
Sep 10 14:32:49.098208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.098171    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-b3cb7d" event={"ID":"0418e94d-7da0-41b8-9ab8-cd45df8d04fb","Type":"ContainerStarted","Data":"e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055"}
Sep 10 14:32:49.098604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.098220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7596-b3cb7d" event={"ID":"0418e94d-7da0-41b8-9ab8-cd45df8d04fb","Type":"ContainerStarted","Data":"78faa3bbf0c5584120710eebb27f39d6a12ce622a0c0a7aa5306f069b6b7fa0a"}
Sep 10 14:32:49.098604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.098354    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7596-d014ac" podUID="38f2b246-b970-46f3-a173-1f248707c846" containerName="test-curl-7596-d014ac" containerID="cri-o://ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600" gracePeriod=2
Sep 10 14:32:49.114458 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.114366    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7596-b3cb7d" podStartSLOduration=2.114351055 podStartE2EDuration="2.114351055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:49.113336855 +0000 UTC m=+1251.201794426" watchObservedRunningTime="2026-09-10 14:32:49.114351055 +0000 UTC m=+1251.202808625"
Sep 10 14:32:49.254542 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.254521    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:49.293066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.293042    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mc9ww\" (UniqueName: \"kubernetes.io/projected/44e9151a-79e3-4e56-b7ef-6af98b137290-kube-api-access-mc9ww\") pod \"44e9151a-79e3-4e56-b7ef-6af98b137290\" (UID: \"44e9151a-79e3-4e56-b7ef-6af98b137290\") "
Sep 10 14:32:49.295220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.295191    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44e9151a-79e3-4e56-b7ef-6af98b137290-kube-api-access-mc9ww" (OuterVolumeSpecName: "kube-api-access-mc9ww") pod "44e9151a-79e3-4e56-b7ef-6af98b137290" (UID: "44e9151a-79e3-4e56-b7ef-6af98b137290"). InnerVolumeSpecName "kube-api-access-mc9ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:49.394052 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.393968    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mc9ww\" (UniqueName: \"kubernetes.io/projected/44e9151a-79e3-4e56-b7ef-6af98b137290-kube-api-access-mc9ww\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:32:49.680051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.680023    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7596-b3cb7d"]
Sep 10 14:32:49.684432 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:49.684402    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7596-b3cb7d"]
Sep 10 14:32:50.103814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.103726    2576 generic.go:356] "Generic (PLEG): container finished" podID="44e9151a-79e3-4e56-b7ef-6af98b137290" containerID="433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2" exitCode=137
Sep 10 14:32:50.103814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.103766    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:50.104299 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.103827    2576 scope.go:117] "RemoveContainer" containerID="433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2"
Sep 10 14:32:50.104299 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:50.104003    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7596-b3cb7d" container="test-curl-7596-b3cb7d"
Sep 10 14:32:50.113057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.113039    2576 scope.go:117] "RemoveContainer" containerID="433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2"
Sep 10 14:32:50.113353 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:50.113336    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2\": container with ID starting with 433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2 not found: ID does not exist" containerID="433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2"
Sep 10 14:32:50.113407 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.113362    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2"} err="failed to get container status \"433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2\": rpc error: code = NotFound desc = could not find container \"433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2\": container with ID starting with 433a9eb41f90b0afe617066f0050dce2c4f5e088eb0d6ca00e616c70bc4f68e2 not found: ID does not exist"
Sep 10 14:32:50.114450 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:50.114429    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7596-89a411\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="44e9151a-79e3-4e56-b7ef-6af98b137290" pod="opendatahub/test-curl-7596-89a411"
Sep 10 14:32:50.305590 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.305558    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:50.428066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.428032    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44e9151a-79e3-4e56-b7ef-6af98b137290" path="/var/lib/kubelet/pods/44e9151a-79e3-4e56-b7ef-6af98b137290/volumes"
Sep 10 14:32:50.531122 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.531088    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"]
Sep 10 14:32:50.531860 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.531835    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" containerName="test-curl-7596-b3cb7d"
Sep 10 14:32:50.532013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.531874    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" containerName="test-curl-7596-b3cb7d"
Sep 10 14:32:50.532013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.531997    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" containerName="test-curl-7596-b3cb7d"
Sep 10 14:32:50.535542 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.535520    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.538060 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.538043    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e96075f7-openshift-default-dockercfg-gkf8g\""
Sep 10 14:32:50.546046 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.546016    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"]
Sep 10 14:32:50.577417 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.577386    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:50.603776 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.603742    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-podinfo\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.603915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.603794    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-credential-socket\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.603915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.603845    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-socket\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.603915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.603902    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-certs\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.604096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.603932    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-token\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.604096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.603992    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-envoy\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.604096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.604038    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csknj\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-kube-api-access-csknj\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.604096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.604080    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-data\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.604340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.604112    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istiod-ca-cert\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.700401 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.700320    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:50.704549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704523    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istiod-ca-cert\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704557    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-podinfo\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704591    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-credential-socket\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704614    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-socket\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704662    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-certs\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704694    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-token\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-envoy\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704754    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csknj\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-kube-api-access-csknj\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.704895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.704794    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-data\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.705096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.705000    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-credential-socket\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.705096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.705081    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-certs\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.705205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.705107    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-socket\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.705205 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.705186    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-data\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.705568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.705546    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istiod-ca-cert\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.707112 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.707086    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-envoy\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.707379 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.707364    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-podinfo\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.713684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.713661    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-token\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.714207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.714181    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csknj\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-kube-api-access-csknj\") pod \"e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") " pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:50.847560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:50.847527    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:51.111716 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.110841    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7596-b3cb7d" podUID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" containerName="test-curl-7596-b3cb7d" containerID="cri-o://e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055" gracePeriod=2
Sep 10 14:32:51.131641 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.131613    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"]
Sep 10 14:32:51.150621 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.150577    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:51.150751 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.150671    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:51.150751 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.150715    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:51.246679 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.246653    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:51.311730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.311694    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8lm7\" (UniqueName: \"kubernetes.io/projected/38f2b246-b970-46f3-a173-1f248707c846-kube-api-access-t8lm7\") pod \"38f2b246-b970-46f3-a173-1f248707c846\" (UID: \"38f2b246-b970-46f3-a173-1f248707c846\") "
Sep 10 14:32:51.313865 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.313841    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38f2b246-b970-46f3-a173-1f248707c846-kube-api-access-t8lm7" (OuterVolumeSpecName: "kube-api-access-t8lm7") pod "38f2b246-b970-46f3-a173-1f248707c846" (UID: "38f2b246-b970-46f3-a173-1f248707c846"). InnerVolumeSpecName "kube-api-access-t8lm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:51.412733 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:51.412637    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8lm7\" (UniqueName: \"kubernetes.io/projected/38f2b246-b970-46f3-a173-1f248707c846-kube-api-access-t8lm7\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:32:52.118126 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.118040    2576 generic.go:356] "Generic (PLEG): container finished" podID="38f2b246-b970-46f3-a173-1f248707c846" containerID="ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600" exitCode=137
Sep 10 14:32:52.118126 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.118083    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:52.118638 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.118153    2576 scope.go:117] "RemoveContainer" containerID="ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600"
Sep 10 14:32:52.120054 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.120024    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" event={"ID":"7b9b0384-58bc-4411-acdb-1af7ee1d22d5","Type":"ContainerStarted","Data":"85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"}
Sep 10 14:32:52.120141 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.120118    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" event={"ID":"7b9b0384-58bc-4411-acdb-1af7ee1d22d5","Type":"ContainerStarted","Data":"c218b1ebc9f71d3e0e6e9cca169a4b2071dd955d64ad28a88f2058602c8570a4"}
Sep 10 14:32:52.127335 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.127320    2576 scope.go:117] "RemoveContainer" containerID="ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600"
Sep 10 14:32:52.127622 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:52.127598    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600\": container with ID starting with ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600 not found: ID does not exist" containerID="ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600"
Sep 10 14:32:52.127692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.127632    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600"} err="failed to get container status \"ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600\": rpc error: code = NotFound desc = could not find container \"ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600\": container with ID starting with ef9ece20f65ba9c4c7cd158fa0696e65b38e49b0f25807a28b8d0437c96b0600 not found: ID does not exist"
Sep 10 14:32:52.139793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.139749    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podStartSLOduration=2.139737939 podStartE2EDuration="2.139737939s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:52.13850036 +0000 UTC m=+1254.226957931" watchObservedRunningTime="2026-09-10 14:32:52.139737939 +0000 UTC m=+1254.228195509"
Sep 10 14:32:52.140655 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:52.140632    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7596-d014ac\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="38f2b246-b970-46f3-a173-1f248707c846" pod="opendatahub/test-curl-7596-d014ac"
Sep 10 14:32:52.428527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.428495    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38f2b246-b970-46f3-a173-1f248707c846" path="/var/lib/kubelet/pods/38f2b246-b970-46f3-a173-1f248707c846/volumes"
Sep 10 14:32:52.848248 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.848143    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:52.853445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:52.853417    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:53.127164 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:53.127140    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:53.128097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:53.128081    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:32:53.291928 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:53.291905    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:53.331518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:53.331489    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjjcm\" (UniqueName: \"kubernetes.io/projected/0418e94d-7da0-41b8-9ab8-cd45df8d04fb-kube-api-access-zjjcm\") pod \"0418e94d-7da0-41b8-9ab8-cd45df8d04fb\" (UID: \"0418e94d-7da0-41b8-9ab8-cd45df8d04fb\") "
Sep 10 14:32:53.333597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:53.333570    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0418e94d-7da0-41b8-9ab8-cd45df8d04fb-kube-api-access-zjjcm" (OuterVolumeSpecName: "kube-api-access-zjjcm") pod "0418e94d-7da0-41b8-9ab8-cd45df8d04fb" (UID: "0418e94d-7da0-41b8-9ab8-cd45df8d04fb"). InnerVolumeSpecName "kube-api-access-zjjcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:53.432792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:53.432766    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjjcm\" (UniqueName: \"kubernetes.io/projected/0418e94d-7da0-41b8-9ab8-cd45df8d04fb-kube-api-access-zjjcm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:32:54.023002 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.022973    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:54.110439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.110403    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:54.132988 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.132962    2576 generic.go:356] "Generic (PLEG): container finished" podID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" containerID="e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055" exitCode=137
Sep 10 14:32:54.133376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.133005    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:54.133376 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.133059    2576 scope.go:117] "RemoveContainer" containerID="e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055"
Sep 10 14:32:54.141981 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.141965    2576 scope.go:117] "RemoveContainer" containerID="e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055"
Sep 10 14:32:54.142267 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:54.142217    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055\": container with ID starting with e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055 not found: ID does not exist" containerID="e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055"
Sep 10 14:32:54.142370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.142269    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055"} err="failed to get container status \"e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055\": rpc error: code = NotFound desc = could not find container \"e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055\": container with ID starting with e2262afb0a3658e9db8739ed599418538620fbfe2a59d30564ed1f29754d2055 not found: ID does not exist"
Sep 10 14:32:54.143639 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:32:54.143618    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7596-b3cb7d\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" pod="opendatahub/test-curl-7596-b3cb7d"
Sep 10 14:32:54.427384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:54.427353    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0418e94d-7da0-41b8-9ab8-cd45df8d04fb" path="/var/lib/kubelet/pods/0418e94d-7da0-41b8-9ab8-cd45df8d04fb/volumes"
Sep 10 14:32:55.173420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:55.173356    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:55.271354 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:55.271306    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:55.372487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:55.372439    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:55.678560 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:55.678499    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:55.781768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:55.781740    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:59.322568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.322539    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:59.808771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.808736    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"]
Sep 10 14:32:59.812513 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.812489    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.816925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.816904    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"kube-root-ca.crt\""
Sep 10 14:32:59.818299 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.818273    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"openshift-service-ca.crt\""
Sep 10 14:32:59.818394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.818295    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"default-dockercfg-vp8v5\""
Sep 10 14:32:59.818394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.818308    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"auth-test-model-586826-kserve-self-signed-certs\""
Sep 10 14:32:59.824390 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.824372    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"]
Sep 10 14:32:59.837223 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.837199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:59.895349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.895313    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk7qj\" (UniqueName: \"kubernetes.io/projected/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-kube-api-access-rk7qj\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.895526 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.895370    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-home\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.895526 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.895452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-dshm\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.895526 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.895496    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-model-cache\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.895697 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.895641    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tmp-dir\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.895746 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.895694    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tls-certs\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996323 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996287    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tmp-dir\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996501 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996341    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tls-certs\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996600 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996578    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk7qj\" (UniqueName: \"kubernetes.io/projected/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-kube-api-access-rk7qj\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996621    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-home\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996638    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-dshm\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996687 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996670    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-model-cache\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996840 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996739    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tmp-dir\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996894 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996873    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-home\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.996979 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.996961    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-model-cache\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.998861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.998836    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-dshm\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:32:59.999154 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:32:59.999134    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tls-certs\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:00.005469 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:00.005447    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk7qj\" (UniqueName: \"kubernetes.io/projected/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-kube-api-access-rk7qj\") pod \"auth-test-model-586826-kserve-cd4548467-zglsx\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") " pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:00.121478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:00.121397    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:00.397517 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:00.397488    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"]
Sep 10 14:33:00.399522 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:33:00.399495    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9c6a5c0c_b0c3_4463_803d_2e2b5b858560.slice/crio-6d1a16a8b32612d55e30d7f0f3b7cd4dc9411a693ac6797b78f9efb53d2802c8 WatchSource:0}: Error finding container 6d1a16a8b32612d55e30d7f0f3b7cd4dc9411a693ac6797b78f9efb53d2802c8: Status 404 returned error can't find the container with id 6d1a16a8b32612d55e30d7f0f3b7cd4dc9411a693ac6797b78f9efb53d2802c8
Sep 10 14:33:01.175321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:01.175276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx" event={"ID":"9c6a5c0c-b0c3-4463-803d-2e2b5b858560","Type":"ContainerStarted","Data":"d3d17ced78519aa591470e6c9a526ea765fd66ea6b604413ac9443250cc359f0"}
Sep 10 14:33:01.175321 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:01.175322    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx" event={"ID":"9c6a5c0c-b0c3-4463-803d-2e2b5b858560","Type":"ContainerStarted","Data":"6d1a16a8b32612d55e30d7f0f3b7cd4dc9411a693ac6797b78f9efb53d2802c8"}
Sep 10 14:33:01.193853 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:01.193802    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx" podStartSLOduration=2.193786425 podStartE2EDuration="2.193786425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:01.192347897 +0000 UTC m=+1263.280805469" watchObservedRunningTime="2026-09-10 14:33:01.193786425 +0000 UTC m=+1263.282243997"
Sep 10 14:33:03.011003 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:03.010964    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:33:10.122112 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:10.122070    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:10.122554 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:10.122163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:10.134883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:10.134860    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:10.235334 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:10.235306    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:33:13.239215 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:13.239183    2576 generic.go:356] "Generic (PLEG): container finished" podID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerID="85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47" exitCode=0
Sep 10 14:33:13.239677 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:13.239266    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" event={"ID":"7b9b0384-58bc-4411-acdb-1af7ee1d22d5","Type":"ContainerDied","Data":"85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"}
Sep 10 14:33:13.239677 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:13.239645    2576 scope.go:117] "RemoveContainer" containerID="85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"
Sep 10 14:33:13.240968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:13.240936    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:33:13.241028 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:13.241008    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:33:13.241075 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:13.241040    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:33:14.246606 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:14.246571    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" event={"ID":"7b9b0384-58bc-4411-acdb-1af7ee1d22d5","Type":"ContainerStarted","Data":"9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a"}
Sep 10 14:33:14.848276 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:14.848222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:33:14.848475 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:14.848454    2576 patch_prober.go:28] interesting pod/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 10 14:33:14.848529 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:14.848501    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 10 14:33:15.848500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:15.848459    2576 patch_prober.go:28] interesting pod/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 10 14:33:15.848993 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:15.848528    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 10 14:33:16.849065 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:16.849023    2576 patch_prober.go:28] interesting pod/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 10 14:33:16.849483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:16.849116    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 10 14:33:17.848387 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:17.848349    2576 patch_prober.go:28] interesting pod/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 10 14:33:17.848586 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:17.848424    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 10 14:33:18.848259 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:18.848202    2576 patch_prober.go:28] interesting pod/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 10 14:33:18.848668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:18.848285    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 10 14:33:19.096164 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:19.096101    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:19.963662 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:19.963628    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:33:19.964148 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:19.963969    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:33:19.964861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:19.964846    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:33:30.425832 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:30.425801    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:35.028197 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:35.028161    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:33:41.160047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:41.160008    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:42.313900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.313867    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"]
Sep 10 14:33:42.316678 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.316659    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.320015 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.319993    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e96075f7\"/\"e2e-lifecycle-model-e96075f7-kserve-self-signed-certs\""
Sep 10 14:33:42.320015 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.320001    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e96075f7\"/\"kube-root-ca.crt\""
Sep 10 14:33:42.320200 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.320015    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-e96075f7\"/\"openshift-service-ca.crt\""
Sep 10 14:33:42.320200 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.320015    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-e96075f7\"/\"default-dockercfg-r2kbz\""
Sep 10 14:33:42.325665 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.325642    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"]
Sep 10 14:33:42.343130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.343104    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:42.409214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.409182    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqs6g\" (UniqueName: \"kubernetes.io/projected/2725a406-da23-4dce-805c-a9b62266f72d-kube-api-access-jqs6g\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.409214 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.409216    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-dshm\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.409423 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.409315    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-home\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.409423 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.409353    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-tmp-dir\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.409423 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.409393    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-model-cache\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.409527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.409428    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2725a406-da23-4dce-805c-a9b62266f72d-tls-certs\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.510733 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.510686    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-tmp-dir\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.510919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.510750    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-model-cache\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.510919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.510781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2725a406-da23-4dce-805c-a9b62266f72d-tls-certs\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.510919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.510836    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqs6g\" (UniqueName: \"kubernetes.io/projected/2725a406-da23-4dce-805c-a9b62266f72d-kube-api-access-jqs6g\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.510919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.510867    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-dshm\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.511071 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.510933    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-home\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.511140 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.511119    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-tmp-dir\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.511207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.511188    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-model-cache\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.511273 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.511254    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-home\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.513726 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.513696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2725a406-da23-4dce-805c-a9b62266f72d-tls-certs\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.513877 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.513857    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-dshm\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.519304 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.519281    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqs6g\" (UniqueName: \"kubernetes.io/projected/2725a406-da23-4dce-805c-a9b62266f72d-kube-api-access-jqs6g\") pod \"e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") " pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.626704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.626612    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:42.898990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:42.898964    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"]
Sep 10 14:33:42.900603 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:33:42.900575    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2725a406_da23_4dce_805c_a9b62266f72d.slice/crio-9f9a14722e31248ce9b388642925233ff955a23dd5c8bd1131f36e5368a57e55 WatchSource:0}: Error finding container 9f9a14722e31248ce9b388642925233ff955a23dd5c8bd1131f36e5368a57e55: Status 404 returned error can't find the container with id 9f9a14722e31248ce9b388642925233ff955a23dd5c8bd1131f36e5368a57e55
Sep 10 14:33:43.409408 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:43.409377    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm" event={"ID":"2725a406-da23-4dce-805c-a9b62266f72d","Type":"ContainerStarted","Data":"9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f"}
Sep 10 14:33:43.409802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:43.409417    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm" event={"ID":"2725a406-da23-4dce-805c-a9b62266f72d","Type":"ContainerStarted","Data":"9f9a14722e31248ce9b388642925233ff955a23dd5c8bd1131f36e5368a57e55"}
Sep 10 14:33:43.428142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:43.428098    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm" podStartSLOduration=1.428081498 podStartE2EDuration="1.428081498s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:33:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:33:43.426511969 +0000 UTC m=+1305.514969543" watchObservedRunningTime="2026-09-10 14:33:43.428081498 +0000 UTC m=+1305.516539076"
Sep 10 14:33:48.316130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:48.316094    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:33:52.627435 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:52.627394    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:52.627899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:52.627451    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:52.639699 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:52.639668    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:53.476664 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:53.476635    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:33:57.208868 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:57.208835    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:57.310208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:33:57.310176    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:05.761146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:05.761108    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:34:21.044895 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:21.044796    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:34:29.274493 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:29.274455    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:39.712220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:39.712181    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:34:44.573126 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:44.573093    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:44.674771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:44.674737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:44.812865 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:44.812830    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:44.937773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:44.937739    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:47.949255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:47.949208    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:52.927269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:52.927217    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"]
Sep 10 14:34:53.022476 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:53.022438    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:58.607394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:34:58.607359    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:35:07.947933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:07.947868    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm" podUID="2725a406-da23-4dce-805c-a9b62266f72d" containerName="main" containerID="cri-o://9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f" gracePeriod=45
Sep 10 14:35:08.203078 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.203020    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:35:08.352878 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.352835    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-tmp-dir\") pod \"2725a406-da23-4dce-805c-a9b62266f72d\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") "
Sep 10 14:35:08.353051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.352931    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-dshm\") pod \"2725a406-da23-4dce-805c-a9b62266f72d\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") "
Sep 10 14:35:08.353051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353012    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-model-cache\") pod \"2725a406-da23-4dce-805c-a9b62266f72d\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") "
Sep 10 14:35:08.353202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353064    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-home\") pod \"2725a406-da23-4dce-805c-a9b62266f72d\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") "
Sep 10 14:35:08.353202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353059    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2725a406-da23-4dce-805c-a9b62266f72d" (UID: "2725a406-da23-4dce-805c-a9b62266f72d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:08.353202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jqs6g\" (UniqueName: \"kubernetes.io/projected/2725a406-da23-4dce-805c-a9b62266f72d-kube-api-access-jqs6g\") pod \"2725a406-da23-4dce-805c-a9b62266f72d\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") "
Sep 10 14:35:08.353202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353134    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2725a406-da23-4dce-805c-a9b62266f72d-tls-certs\") pod \"2725a406-da23-4dce-805c-a9b62266f72d\" (UID: \"2725a406-da23-4dce-805c-a9b62266f72d\") "
Sep 10 14:35:08.353424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353254    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-model-cache" (OuterVolumeSpecName: "model-cache") pod "2725a406-da23-4dce-805c-a9b62266f72d" (UID: "2725a406-da23-4dce-805c-a9b62266f72d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:08.353502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353482    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:08.353564 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353536    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:08.353654 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.353616    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-home" (OuterVolumeSpecName: "home") pod "2725a406-da23-4dce-805c-a9b62266f72d" (UID: "2725a406-da23-4dce-805c-a9b62266f72d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:08.355615 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.355582    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2725a406-da23-4dce-805c-a9b62266f72d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2725a406-da23-4dce-805c-a9b62266f72d" (UID: "2725a406-da23-4dce-805c-a9b62266f72d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:35:08.355738 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.355663    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-dshm" (OuterVolumeSpecName: "dshm") pod "2725a406-da23-4dce-805c-a9b62266f72d" (UID: "2725a406-da23-4dce-805c-a9b62266f72d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:08.355738 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.355709    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2725a406-da23-4dce-805c-a9b62266f72d-kube-api-access-jqs6g" (OuterVolumeSpecName: "kube-api-access-jqs6g") pod "2725a406-da23-4dce-805c-a9b62266f72d" (UID: "2725a406-da23-4dce-805c-a9b62266f72d"). InnerVolumeSpecName "kube-api-access-jqs6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:08.454322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.454226    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:08.454322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.454278    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jqs6g\" (UniqueName: \"kubernetes.io/projected/2725a406-da23-4dce-805c-a9b62266f72d-kube-api-access-jqs6g\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:08.454322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.454295    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2725a406-da23-4dce-805c-a9b62266f72d-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:08.454322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.454308    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2725a406-da23-4dce-805c-a9b62266f72d-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:08.847203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.847103    2576 generic.go:356] "Generic (PLEG): container finished" podID="2725a406-da23-4dce-805c-a9b62266f72d" containerID="9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f" exitCode=0
Sep 10 14:35:08.847203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.847166    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm" event={"ID":"2725a406-da23-4dce-805c-a9b62266f72d","Type":"ContainerDied","Data":"9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f"}
Sep 10 14:35:08.847203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.847183    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"
Sep 10 14:35:08.847493 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.847210    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm" event={"ID":"2725a406-da23-4dce-805c-a9b62266f72d","Type":"ContainerDied","Data":"9f9a14722e31248ce9b388642925233ff955a23dd5c8bd1131f36e5368a57e55"}
Sep 10 14:35:08.847493 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.847249    2576 scope.go:117] "RemoveContainer" containerID="9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f"
Sep 10 14:35:08.857481 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.857461    2576 scope.go:117] "RemoveContainer" containerID="9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f"
Sep 10 14:35:08.857789 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:35:08.857768    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f\": container with ID starting with 9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f not found: ID does not exist" containerID="9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f"
Sep 10 14:35:08.857862 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.857800    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f"} err="failed to get container status \"9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f\": rpc error: code = NotFound desc = could not find container \"9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f\": container with ID starting with 9d51167d8e2750aa67d5c8c4fa39cdeda0366c0d227a188077bf1c16ad39ec4f not found: ID does not exist"
Sep 10 14:35:08.863269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.863248    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"]
Sep 10 14:35:08.867160 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:08.867128    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-e96075f7/e2e-lifecycle-model-e96075f7-kserve-57b4cb6d75-bs2vm"]
Sep 10 14:35:10.427042 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:10.427008    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2725a406-da23-4dce-805c-a9b62266f72d" path="/var/lib/kubelet/pods/2725a406-da23-4dce-805c-a9b62266f72d/volumes"
Sep 10 14:35:14.229315 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:14.229276    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:14.611611 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:14.611528    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:35:30.012046 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:30.012014    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:30.157504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:30.157469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:30.415467 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:30.415386    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:35:35.930599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:35.930559    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:36.037141 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:36.037049    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:36.160027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:36.159982    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"]
Sep 10 14:35:36.160307 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:36.160274    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy" containerID="cri-o://9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a" gracePeriod=30
Sep 10 14:35:36.185957 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:36.185928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:37.424200 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:37.424163    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:37.532508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:37.532457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:37.626784 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:37.626755    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:38.181881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:38.181844    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:38.285884 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:38.285849    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:38.385909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:38.385873    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:39.574166 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:39.574126    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:41.412308 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.412281    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:35:41.557783 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557677    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-data\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.557783 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557749    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istiod-ca-cert\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.557783 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557778    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-token\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557815    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-credential-socket\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557873    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-certs\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557908    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557933    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-socket\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.557959    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csknj\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-kube-api-access-csknj\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558005    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-envoy\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558032    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-podinfo\") pod \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\" (UID: \"7b9b0384-58bc-4411-acdb-1af7ee1d22d5\") "
Sep 10 14:35:41.558452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558066    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:41.558452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558115    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:41.558452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558146    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:35:41.558452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558307    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:41.558452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558427    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.558452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558448    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.558793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558461    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.558793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558477    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.558793 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.558509    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.560383 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.560355    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-kube-api-access-csknj" (OuterVolumeSpecName: "kube-api-access-csknj") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "kube-api-access-csknj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:41.560515 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.560396    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:35:41.560585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.560546    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:41.560680 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.560655    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7b9b0384-58bc-4411-acdb-1af7ee1d22d5" (UID: "7b9b0384-58bc-4411-acdb-1af7ee1d22d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:35:41.659131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.659095    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csknj\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-kube-api-access-csknj\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.659131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.659123    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.659131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.659132    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:41.659131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:41.659147    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b9b0384-58bc-4411-acdb-1af7ee1d22d5-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:42.024331 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.024295    2576 generic.go:356] "Generic (PLEG): container finished" podID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerID="9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a" exitCode=0
Sep 10 14:35:42.024504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.024345    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"
Sep 10 14:35:42.024504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.024356    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" event={"ID":"7b9b0384-58bc-4411-acdb-1af7ee1d22d5","Type":"ContainerDied","Data":"9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a"}
Sep 10 14:35:42.024504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.024401    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx" event={"ID":"7b9b0384-58bc-4411-acdb-1af7ee1d22d5","Type":"ContainerDied","Data":"c218b1ebc9f71d3e0e6e9cca169a4b2071dd955d64ad28a88f2058602c8570a4"}
Sep 10 14:35:42.024504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.024422    2576 scope.go:117] "RemoveContainer" containerID="9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a"
Sep 10 14:35:42.035806 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.035782    2576 scope.go:117] "RemoveContainer" containerID="85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"
Sep 10 14:35:42.044549 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.044528    2576 scope.go:117] "RemoveContainer" containerID="9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a"
Sep 10 14:35:42.044822 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:35:42.044802    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a\": container with ID starting with 9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a not found: ID does not exist" containerID="9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a"
Sep 10 14:35:42.044864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.044832    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a"} err="failed to get container status \"9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a\": rpc error: code = NotFound desc = could not find container \"9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a\": container with ID starting with 9d047f8a2278da0d04b6ee896e6459cf7319e650fe0adfbf2b8ff27e0b38c86a not found: ID does not exist"
Sep 10 14:35:42.044864 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.044849    2576 scope.go:117] "RemoveContainer" containerID="85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"
Sep 10 14:35:42.045115 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:35:42.045095    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47\": container with ID starting with 85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47 not found: ID does not exist" containerID="85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"
Sep 10 14:35:42.045176 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.045119    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47"} err="failed to get container status \"85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47\": rpc error: code = NotFound desc = could not find container \"85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47\": container with ID starting with 85e485f60cbf0276aee8c26517e1d6acbac8687829a9195bebfef6fe01229f47 not found: ID does not exist"
Sep 10 14:35:42.057923 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.057894    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"]
Sep 10 14:35:42.074840 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.074811    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e96075f7-openshift-default-6d67d46b64-l86jx"]
Sep 10 14:35:42.427604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.427571    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" path="/var/lib/kubelet/pods/7b9b0384-58bc-4411-acdb-1af7ee1d22d5/volumes"
Sep 10 14:35:42.468758 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.468726    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7599-d3e331"]
Sep 10 14:35:42.469097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.469083    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy"
Sep 10 14:35:42.469097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.469098    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy"
Sep 10 14:35:42.469190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.469124    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2725a406-da23-4dce-805c-a9b62266f72d" containerName="main"
Sep 10 14:35:42.469190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.469130    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2725a406-da23-4dce-805c-a9b62266f72d" containerName="main"
Sep 10 14:35:42.469190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.469182    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy"
Sep 10 14:35:42.469190 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.469191    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2725a406-da23-4dce-805c-a9b62266f72d" containerName="main"
Sep 10 14:35:42.472197 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.472180    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:42.475081 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.475059    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2hkkb\""
Sep 10 14:35:42.477416 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.477394    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7599-d3e331"]
Sep 10 14:35:42.568253 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.568204    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf47d\" (UniqueName: \"kubernetes.io/projected/70445138-82c1-4594-b593-30366c534385-kube-api-access-nf47d\") pod \"test-curl-7599-d3e331\" (UID: \"70445138-82c1-4594-b593-30366c534385\") " pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:42.668745 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.668711    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf47d\" (UniqueName: \"kubernetes.io/projected/70445138-82c1-4594-b593-30366c534385-kube-api-access-nf47d\") pod \"test-curl-7599-d3e331\" (UID: \"70445138-82c1-4594-b593-30366c534385\") " pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:42.676913 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.676872    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf47d\" (UniqueName: \"kubernetes.io/projected/70445138-82c1-4594-b593-30366c534385-kube-api-access-nf47d\") pod \"test-curl-7599-d3e331\" (UID: \"70445138-82c1-4594-b593-30366c534385\") " pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:42.782835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:42.782756    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:43.054825 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:43.054742    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7599-d3e331"]
Sep 10 14:35:43.058721 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:35:43.058688    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod70445138_82c1_4594_b593_30366c534385.slice/crio-857757a360320c3f9549cc98370877b37ea3fe20034ede14527aaa10c227b373 WatchSource:0}: Error finding container 857757a360320c3f9549cc98370877b37ea3fe20034ede14527aaa10c227b373: Status 404 returned error can't find the container with id 857757a360320c3f9549cc98370877b37ea3fe20034ede14527aaa10c227b373
Sep 10 14:35:44.038327 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:44.038293    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7599-d3e331" event={"ID":"70445138-82c1-4594-b593-30366c534385","Type":"ContainerStarted","Data":"0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491"}
Sep 10 14:35:44.038701 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:44.038335    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7599-d3e331" event={"ID":"70445138-82c1-4594-b593-30366c534385","Type":"ContainerStarted","Data":"857757a360320c3f9549cc98370877b37ea3fe20034ede14527aaa10c227b373"}
Sep 10 14:35:44.056420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:44.056360    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7599-d3e331" podStartSLOduration=2.056338966 podStartE2EDuration="2.056338966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:44.053186435 +0000 UTC m=+1426.141644009" watchObservedRunningTime="2026-09-10 14:35:44.056338966 +0000 UTC m=+1426.144796537"
Sep 10 14:35:44.649456 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:44.649420    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7599-d3e331"]
Sep 10 14:35:44.653926 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:44.653894    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7599-d3e331"]
Sep 10 14:35:45.043038 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:35:45.043006    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7599-d3e331" container="test-curl-7599-d3e331"
Sep 10 14:35:45.440578 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:45.440551    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:45.540546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:45.540513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:45.951965 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:45.951930    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"]
Sep 10 14:35:45.973599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:45.973574    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:46.048270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:46.048216    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7599-d3e331" podUID="70445138-82c1-4594-b593-30366c534385" containerName="test-curl-7599-d3e331" containerID="cri-o://0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491" gracePeriod=2
Sep 10 14:35:46.050769 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:35:46.050708    2576 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7599-d3e331\" is forbidden: User \"system:node:ip-10-0-128-41.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-41.ec2.internal' and this object" podUID="70445138-82c1-4594-b593-30366c534385" pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:46.106692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:46.106655    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:47.161278 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:47.161216    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:48.239058 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:48.239031    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:48.319724 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:48.319636    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf47d\" (UniqueName: \"kubernetes.io/projected/70445138-82c1-4594-b593-30366c534385-kube-api-access-nf47d\") pod \"70445138-82c1-4594-b593-30366c534385\" (UID: \"70445138-82c1-4594-b593-30366c534385\") "
Sep 10 14:35:48.321775 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:48.321753    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70445138-82c1-4594-b593-30366c534385-kube-api-access-nf47d" (OuterVolumeSpecName: "kube-api-access-nf47d") pod "70445138-82c1-4594-b593-30366c534385" (UID: "70445138-82c1-4594-b593-30366c534385"). InnerVolumeSpecName "kube-api-access-nf47d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:48.420802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:48.420765    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf47d\" (UniqueName: \"kubernetes.io/projected/70445138-82c1-4594-b593-30366c534385-kube-api-access-nf47d\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:35:48.427302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:48.427269    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70445138-82c1-4594-b593-30366c534385" path="/var/lib/kubelet/pods/70445138-82c1-4594-b593-30366c534385/volumes"
Sep 10 14:35:49.066074 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:49.065991    2576 generic.go:356] "Generic (PLEG): container finished" podID="70445138-82c1-4594-b593-30366c534385" containerID="0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491" exitCode=137
Sep 10 14:35:49.066074 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:49.066037    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-d3e331"
Sep 10 14:35:49.066303 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:49.066099    2576 scope.go:117] "RemoveContainer" containerID="0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491"
Sep 10 14:35:49.074514 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:49.074495    2576 scope.go:117] "RemoveContainer" containerID="0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491"
Sep 10 14:35:49.074800 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:35:49.074780    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491\": container with ID starting with 0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491 not found: ID does not exist" containerID="0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491"
Sep 10 14:35:49.074859 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:49.074812    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491"} err="failed to get container status \"0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491\": rpc error: code = NotFound desc = could not find container \"0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491\": container with ID starting with 0e519fa59a94789f1e27a03144b7a1facf11151c46a8e351bf739513e0b32491 not found: ID does not exist"
Sep 10 14:35:51.018351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:51.018315    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:35:52.173460 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:52.173428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:52.298500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:52.298465    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:52.543269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:52.543165    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:57.766544 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:57.766502    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:57.822089 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:35:57.822046    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:00.974028 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:00.973973    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx" podUID="9c6a5c0c-b0c3-4463-803d-2e2b5b858560" containerName="main" containerID="cri-o://d3d17ced78519aa591470e6c9a526ea765fd66ea6b604413ac9443250cc359f0" gracePeriod=45
Sep 10 14:36:01.132387 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.132357    2576 generic.go:356] "Generic (PLEG): container finished" podID="9c6a5c0c-b0c3-4463-803d-2e2b5b858560" containerID="d3d17ced78519aa591470e6c9a526ea765fd66ea6b604413ac9443250cc359f0" exitCode=0
Sep 10 14:36:01.132634 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.132428    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx" event={"ID":"9c6a5c0c-b0c3-4463-803d-2e2b5b858560","Type":"ContainerDied","Data":"d3d17ced78519aa591470e6c9a526ea765fd66ea6b604413ac9443250cc359f0"}
Sep 10 14:36:01.242009 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.241948    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:36:01.438537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438486    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tls-certs\") pod \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") "
Sep 10 14:36:01.438537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438547    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tmp-dir\") pod \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") "
Sep 10 14:36:01.438802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438570    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-model-cache\") pod \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") "
Sep 10 14:36:01.438802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438588    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk7qj\" (UniqueName: \"kubernetes.io/projected/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-kube-api-access-rk7qj\") pod \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") "
Sep 10 14:36:01.438802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438627    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-home\") pod \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") "
Sep 10 14:36:01.438802 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438642    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-dshm\") pod \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\" (UID: \"9c6a5c0c-b0c3-4463-803d-2e2b5b858560\") "
Sep 10 14:36:01.439022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438833    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9c6a5c0c-b0c3-4463-803d-2e2b5b858560" (UID: "9c6a5c0c-b0c3-4463-803d-2e2b5b858560"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:01.439022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438885    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-model-cache" (OuterVolumeSpecName: "model-cache") pod "9c6a5c0c-b0c3-4463-803d-2e2b5b858560" (UID: "9c6a5c0c-b0c3-4463-803d-2e2b5b858560"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:01.439022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.438946    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-home" (OuterVolumeSpecName: "home") pod "9c6a5c0c-b0c3-4463-803d-2e2b5b858560" (UID: "9c6a5c0c-b0c3-4463-803d-2e2b5b858560"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:01.439022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.439010    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:36:01.439171 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.439031    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:36:01.439171 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.439050    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:36:01.440910 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.440887    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-dshm" (OuterVolumeSpecName: "dshm") pod "9c6a5c0c-b0c3-4463-803d-2e2b5b858560" (UID: "9c6a5c0c-b0c3-4463-803d-2e2b5b858560"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:36:01.440992 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.440950    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9c6a5c0c-b0c3-4463-803d-2e2b5b858560" (UID: "9c6a5c0c-b0c3-4463-803d-2e2b5b858560"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:36:01.441052 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.441028    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-kube-api-access-rk7qj" (OuterVolumeSpecName: "kube-api-access-rk7qj") pod "9c6a5c0c-b0c3-4463-803d-2e2b5b858560" (UID: "9c6a5c0c-b0c3-4463-803d-2e2b5b858560"). InnerVolumeSpecName "kube-api-access-rk7qj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:36:01.539697 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.539604    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:36:01.539697 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.539640    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk7qj\" (UniqueName: \"kubernetes.io/projected/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-kube-api-access-rk7qj\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:36:01.539697 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:01.539651    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9c6a5c0c-b0c3-4463-803d-2e2b5b858560-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:36:02.138329 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.138289    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx" event={"ID":"9c6a5c0c-b0c3-4463-803d-2e2b5b858560","Type":"ContainerDied","Data":"6d1a16a8b32612d55e30d7f0f3b7cd4dc9411a693ac6797b78f9efb53d2802c8"}
Sep 10 14:36:02.138329 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.138324    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"
Sep 10 14:36:02.138848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.138345    2576 scope.go:117] "RemoveContainer" containerID="d3d17ced78519aa591470e6c9a526ea765fd66ea6b604413ac9443250cc359f0"
Sep 10 14:36:02.159974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.159944    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"]
Sep 10 14:36:02.164643 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.164615    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/auth-test-model-586826-kserve-cd4548467-zglsx"]
Sep 10 14:36:02.428453 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.428418    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c6a5c0c-b0c3-4463-803d-2e2b5b858560" path="/var/lib/kubelet/pods/9c6a5c0c-b0c3-4463-803d-2e2b5b858560/volumes"
Sep 10 14:36:02.925851 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:02.925820    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:03.160191 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:03.160154    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:05.206070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206042    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"]
Sep 10 14:36:05.206508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206441    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy"
Sep 10 14:36:05.206508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206453    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy"
Sep 10 14:36:05.206508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206465    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c6a5c0c-b0c3-4463-803d-2e2b5b858560" containerName="main"
Sep 10 14:36:05.206508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206470    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c6a5c0c-b0c3-4463-803d-2e2b5b858560" containerName="main"
Sep 10 14:36:05.206508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206482    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70445138-82c1-4594-b593-30366c534385" containerName="test-curl-7599-d3e331"
Sep 10 14:36:05.206508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206487    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="70445138-82c1-4594-b593-30366c534385" containerName="test-curl-7599-d3e331"
Sep 10 14:36:05.207105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206543    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b9b0384-58bc-4411-acdb-1af7ee1d22d5" containerName="istio-proxy"
Sep 10 14:36:05.207105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206552    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="70445138-82c1-4594-b593-30366c534385" containerName="test-curl-7599-d3e331"
Sep 10 14:36:05.207105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.206560    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c6a5c0c-b0c3-4463-803d-2e2b5b858560" containerName="main"
Sep 10 14:36:05.209858 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.209835    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.213405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.213364    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"openshift-service-ca.crt\""
Sep 10 14:36:05.213405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.213404    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"default-dockercfg-vp8v5\""
Sep 10 14:36:05.213637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.213405    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"kube-root-ca.crt\""
Sep 10 14:36:05.213637 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.213429    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"sub-test-model-586826-kserve-self-signed-certs\""
Sep 10 14:36:05.217632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.217610    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"]
Sep 10 14:36:05.229812 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.229787    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:05.272390 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.272361    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-model-cache\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.272521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.272418    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-home\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.272521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.272491    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-dshm\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.272599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.272519    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cndtx\" (UniqueName: \"kubernetes.io/projected/549684a6-e39f-4f1f-99ed-46cddcf8e794-kube-api-access-cndtx\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.272599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.272543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/549684a6-e39f-4f1f-99ed-46cddcf8e794-tls-certs\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.272599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.272563    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-tmp-dir\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.372998 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.372964    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-home\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373172 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373016    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-dshm\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373172 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373140    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cndtx\" (UniqueName: \"kubernetes.io/projected/549684a6-e39f-4f1f-99ed-46cddcf8e794-kube-api-access-cndtx\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373303 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373179    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/549684a6-e39f-4f1f-99ed-46cddcf8e794-tls-certs\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373372 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373321    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-tmp-dir\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373390    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-home\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373495 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373442    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-model-cache\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373637    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-tmp-dir\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.373732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.373715    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-model-cache\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.375465 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.375442    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-dshm\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.375792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.375773    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/549684a6-e39f-4f1f-99ed-46cddcf8e794-tls-certs\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.381040 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.381016    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cndtx\" (UniqueName: \"kubernetes.io/projected/549684a6-e39f-4f1f-99ed-46cddcf8e794-kube-api-access-cndtx\") pod \"sub-test-model-586826-kserve-5d9b9c97d5-h68hn\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") " pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.521191 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.521101    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:05.792723 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:05.792700    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"]
Sep 10 14:36:05.794960 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:36:05.794936    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod549684a6_e39f_4f1f_99ed_46cddcf8e794.slice/crio-ac6872785e1b37d51d3166d48b6b6fa5179e03505a328c09bdf1361a3cae8395 WatchSource:0}: Error finding container ac6872785e1b37d51d3166d48b6b6fa5179e03505a328c09bdf1361a3cae8395: Status 404 returned error can't find the container with id ac6872785e1b37d51d3166d48b6b6fa5179e03505a328c09bdf1361a3cae8395
Sep 10 14:36:06.162099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:06.162009    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn" event={"ID":"549684a6-e39f-4f1f-99ed-46cddcf8e794","Type":"ContainerStarted","Data":"12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559"}
Sep 10 14:36:06.162099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:06.162056    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn" event={"ID":"549684a6-e39f-4f1f-99ed-46cddcf8e794","Type":"ContainerStarted","Data":"ac6872785e1b37d51d3166d48b6b6fa5179e03505a328c09bdf1361a3cae8395"}
Sep 10 14:36:06.178345 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:06.178294    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn" podStartSLOduration=1.178279022 podStartE2EDuration="1.178279022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:06.177192661 +0000 UTC m=+1448.265650258" watchObservedRunningTime="2026-09-10 14:36:06.178279022 +0000 UTC m=+1448.266736594"
Sep 10 14:36:09.513453 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:09.513416    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:36:15.521781 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:15.521745    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:15.522310 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:15.521793    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:15.534387 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:15.534363    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:16.223488 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:16.223462    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:36:19.908203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.908163    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"]
Sep 10 14:36:19.911518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.911494    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:19.914047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.914025    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"openshift-service-ca.crt\""
Sep 10 14:36:19.915077 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.915054    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"sub-test-model-9dcc39-kserve-self-signed-certs\""
Sep 10 14:36:19.915077 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.915080    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"default-dockercfg-442hr\""
Sep 10 14:36:19.915226 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.915161    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-9dcc39\"/\"kube-root-ca.crt\""
Sep 10 14:36:19.920742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.920720    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"]
Sep 10 14:36:19.940300 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:19.940267    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:20.001900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.001861    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-model-cache\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.001900 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.001897    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tls-certs\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.002156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.001928    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tmp-dir\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.002156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.002034    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-dshm\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.002156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.002084    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxjql\" (UniqueName: \"kubernetes.io/projected/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-kube-api-access-pxjql\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.002156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.002139    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-home\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.102882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.102833    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-model-cache\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.102882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.102880    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tls-certs\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.102911    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tmp-dir\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.102945    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-dshm\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.102962    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxjql\" (UniqueName: \"kubernetes.io/projected/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-kube-api-access-pxjql\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.102999    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-home\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103396 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.103339    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-model-cache\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.103386    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tmp-dir\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.103439 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.103421    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-home\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.105510 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.105488    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-dshm\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.105663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.105645    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tls-certs\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.111372 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.111345    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxjql\" (UniqueName: \"kubernetes.io/projected/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-kube-api-access-pxjql\") pod \"sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") " pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.222673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.222636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:20.493034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:20.492970    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"]
Sep 10 14:36:20.495252 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:36:20.495202    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd6aa7ae_a5ff_440c_a56f_0af585a7e47a.slice/crio-f1ed01f11f328c02058e8262e88128949fbe82a51dcc1f2c2b9af1568e16d107 WatchSource:0}: Error finding container f1ed01f11f328c02058e8262e88128949fbe82a51dcc1f2c2b9af1568e16d107: Status 404 returned error can't find the container with id f1ed01f11f328c02058e8262e88128949fbe82a51dcc1f2c2b9af1568e16d107
Sep 10 14:36:21.238576 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:21.238534    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq" event={"ID":"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a","Type":"ContainerStarted","Data":"27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e"}
Sep 10 14:36:21.238576 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:21.238583    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq" event={"ID":"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a","Type":"ContainerStarted","Data":"f1ed01f11f328c02058e8262e88128949fbe82a51dcc1f2c2b9af1568e16d107"}
Sep 10 14:36:21.254759 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:21.254675    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq" podStartSLOduration=2.254655004 podStartE2EDuration="2.254655004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:21.253635459 +0000 UTC m=+1463.342093032" watchObservedRunningTime="2026-09-10 14:36:21.254655004 +0000 UTC m=+1463.343112576"
Sep 10 14:36:24.089216 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:24.089173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:25.507392 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:25.507356    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:36:29.935478 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:29.935412    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:29.999403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:29.999376    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:30.222801 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:30.222712    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:30.222801 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:30.222766    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:30.235917 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:30.235890    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:30.273628 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:30.273581    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:30.301110 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:30.301084    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:36:30.566809 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:30.566725    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:31.061136 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:31.061108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:33.725027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:33.724988    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:33.825384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:33.825347    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:34.923503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:34.923463    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:35.023701 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:35.023665    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:35.302508 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:35.302422    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:35.407450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:35.407381    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:35.480651 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:35.480613    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:39.526691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:39.526660    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:36:55.516543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:55.516506    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:36:58.419411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:58.419382    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:36:58.445461 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:58.445436    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:36:58.621448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:58.621408    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:59.038208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.038173    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:59.321470 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.321392    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"]
Sep 10 14:36:59.324897 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.324879    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.327575 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.327549    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7ae247-openshift-default-dockercfg-nnnk9\""
Sep 10 14:36:59.336735 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.336713    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"]
Sep 10 14:36:59.355279 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.355221    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:59.472361 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472326    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a658b53-8af1-4a91-a52f-630d979de032-istiod-ca-cert\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472390    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-certs\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472452    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-istio-token\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472489    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsrdx\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-kube-api-access-bsrdx\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472523    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a658b53-8af1-4a91-a52f-630d979de032-istio-podinfo\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472554    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-envoy\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472578    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-data\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472632    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-credential-socket\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.472837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.472678    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-socket\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574082 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.573988    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a658b53-8af1-4a91-a52f-630d979de032-istiod-ca-cert\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574082 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574061    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-certs\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574107    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-istio-token\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574145    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsrdx\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-kube-api-access-bsrdx\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574181    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a658b53-8af1-4a91-a52f-630d979de032-istio-podinfo\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574211    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-envoy\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574254    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-data\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574302    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-credential-socket\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574651 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574361    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-socket\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574651 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574473    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-certs\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574695    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-data\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574768 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574748    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-credential-socket\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.574877 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.574826    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-socket\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.575324 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.575255    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a658b53-8af1-4a91-a52f-630d979de032-istiod-ca-cert\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.576670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.576650    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-envoy\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.576928 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.576905    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a658b53-8af1-4a91-a52f-630d979de032-istio-podinfo\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.582716 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.582691    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-istio-token\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.583308 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.583290    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsrdx\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-kube-api-access-bsrdx\") pod \"e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") " pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.638283 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.638224    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:36:59.695454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.695205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:59.942039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.942011    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"]
Sep 10 14:36:59.945364 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:36:59.945332    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a658b53_8af1_4a91_a52f_630d979de032.slice/crio-77f539ac5292514026c781063f48ad700ee282377fbd4da4cfb4a33ccdf450b7 WatchSource:0}: Error finding container 77f539ac5292514026c781063f48ad700ee282377fbd4da4cfb4a33ccdf450b7: Status 404 returned error can't find the container with id 77f539ac5292514026c781063f48ad700ee282377fbd4da4cfb4a33ccdf450b7
Sep 10 14:36:59.947899 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.947865    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:36:59.947962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.947934    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:36:59.947999 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:36:59.947962    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:37:00.451538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:00.451493    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" event={"ID":"6a658b53-8af1-4a91-a52f-630d979de032","Type":"ContainerStarted","Data":"72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"}
Sep 10 14:37:00.451713 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:00.451556    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" event={"ID":"6a658b53-8af1-4a91-a52f-630d979de032","Type":"ContainerStarted","Data":"77f539ac5292514026c781063f48ad700ee282377fbd4da4cfb4a33ccdf450b7"}
Sep 10 14:37:00.473358 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:00.473302    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podStartSLOduration=1.4732852 podStartE2EDuration="1.4732852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:00.47138537 +0000 UTC m=+1502.559842942" watchObservedRunningTime="2026-09-10 14:37:00.4732852 +0000 UTC m=+1502.561742770"
Sep 10 14:37:00.638494 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:00.638459    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:01.644401 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:01.644370    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:02.367961 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:02.367924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:02.462523 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:02.462482    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:02.463411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:02.463390    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:02.466805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:02.466782    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:03.499843 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:03.499809    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:03.665566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:03.665530    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:03.908845 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:03.908724    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:04.021771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:04.021737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:04.121088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:04.121052    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:06.504158 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.504128    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:06.741535 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.741499    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7599-176b2a"]
Sep 10 14:37:06.745134 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.745116    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:06.748116 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.748089    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-2hkkb\""
Sep 10 14:37:06.750159 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.750138    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7599-176b2a"]
Sep 10 14:37:06.845157 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.845124    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xszlv\" (UniqueName: \"kubernetes.io/projected/b2b4a96e-425b-4f01-9406-581823b96030-kube-api-access-xszlv\") pod \"test-curl-7599-176b2a\" (UID: \"b2b4a96e-425b-4f01-9406-581823b96030\") " pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:06.946339 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.946303    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xszlv\" (UniqueName: \"kubernetes.io/projected/b2b4a96e-425b-4f01-9406-581823b96030-kube-api-access-xszlv\") pod \"test-curl-7599-176b2a\" (UID: \"b2b4a96e-425b-4f01-9406-581823b96030\") " pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:06.954698 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:06.954669    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xszlv\" (UniqueName: \"kubernetes.io/projected/b2b4a96e-425b-4f01-9406-581823b96030-kube-api-access-xszlv\") pod \"test-curl-7599-176b2a\" (UID: \"b2b4a96e-425b-4f01-9406-581823b96030\") " pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:07.057975 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:07.057872    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:07.332700 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:07.332554    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7599-176b2a"]
Sep 10 14:37:07.335743 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:37:07.335714    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb2b4a96e_425b_4f01_9406_581823b96030.slice/crio-c2e33a88e27ad98d30fc6e03c614aa9fcee9de9b74df508fed0bc38ff4d7959c WatchSource:0}: Error finding container c2e33a88e27ad98d30fc6e03c614aa9fcee9de9b74df508fed0bc38ff4d7959c: Status 404 returned error can't find the container with id c2e33a88e27ad98d30fc6e03c614aa9fcee9de9b74df508fed0bc38ff4d7959c
Sep 10 14:37:07.489546 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:07.489511    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7599-176b2a" event={"ID":"b2b4a96e-425b-4f01-9406-581823b96030","Type":"ContainerStarted","Data":"203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222"}
Sep 10 14:37:07.489737 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:07.489555    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7599-176b2a" event={"ID":"b2b4a96e-425b-4f01-9406-581823b96030","Type":"ContainerStarted","Data":"c2e33a88e27ad98d30fc6e03c614aa9fcee9de9b74df508fed0bc38ff4d7959c"}
Sep 10 14:37:07.510025 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:07.509974    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7599-176b2a" podStartSLOduration=1.5099565560000001 podStartE2EDuration="1.509956556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:07.50819141 +0000 UTC m=+1509.596648980" watchObservedRunningTime="2026-09-10 14:37:07.509956556 +0000 UTC m=+1509.598414127"
Sep 10 14:37:08.088119 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.088085    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7599-176b2a"]
Sep 10 14:37:08.097497 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.096493    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7599-176b2a"]
Sep 10 14:37:08.204785 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.204751    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7599-b3cff5"]
Sep 10 14:37:08.205167 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.205154    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b2b4a96e-425b-4f01-9406-581823b96030" containerName="test-curl-7599-176b2a"
Sep 10 14:37:08.205217 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.205168    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b2b4a96e-425b-4f01-9406-581823b96030" containerName="test-curl-7599-176b2a"
Sep 10 14:37:08.205284 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.205261    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b2b4a96e-425b-4f01-9406-581823b96030" containerName="test-curl-7599-176b2a"
Sep 10 14:37:08.208421 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.208405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:08.214308 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.214282    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7599-b3cff5"]
Sep 10 14:37:08.257652 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.257619    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w8z7\" (UniqueName: \"kubernetes.io/projected/d8b274e2-095b-4c14-85f8-2f950fa685c4-kube-api-access-7w8z7\") pod \"test-curl-7599-b3cff5\" (UID: \"d8b274e2-095b-4c14-85f8-2f950fa685c4\") " pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:08.359044 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.358946    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w8z7\" (UniqueName: \"kubernetes.io/projected/d8b274e2-095b-4c14-85f8-2f950fa685c4-kube-api-access-7w8z7\") pod \"test-curl-7599-b3cff5\" (UID: \"d8b274e2-095b-4c14-85f8-2f950fa685c4\") " pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:08.367016 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.366994    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w8z7\" (UniqueName: \"kubernetes.io/projected/d8b274e2-095b-4c14-85f8-2f950fa685c4-kube-api-access-7w8z7\") pod \"test-curl-7599-b3cff5\" (UID: \"d8b274e2-095b-4c14-85f8-2f950fa685c4\") " pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:08.494772 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:37:08.494743    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7599-176b2a" container="test-curl-7599-176b2a"
Sep 10 14:37:08.518808 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.518787    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:08.789792 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:08.789765    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7599-b3cff5"]
Sep 10 14:37:08.792223 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:37:08.792195    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd8b274e2_095b_4c14_85f8_2f950fa685c4.slice/crio-ec4e56e3df78b3601d39ed377fc46fdb23c1a7880414a6250337dc01fbde9a68 WatchSource:0}: Error finding container ec4e56e3df78b3601d39ed377fc46fdb23c1a7880414a6250337dc01fbde9a68: Status 404 returned error can't find the container with id ec4e56e3df78b3601d39ed377fc46fdb23c1a7880414a6250337dc01fbde9a68
Sep 10 14:37:09.501669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:09.501632    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7599-b3cff5" event={"ID":"d8b274e2-095b-4c14-85f8-2f950fa685c4","Type":"ContainerStarted","Data":"7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d"}
Sep 10 14:37:09.501669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:09.501661    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7599-176b2a" podUID="b2b4a96e-425b-4f01-9406-581823b96030" containerName="test-curl-7599-176b2a" containerID="cri-o://203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222" gracePeriod=2
Sep 10 14:37:09.501669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:09.501681    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7599-b3cff5" event={"ID":"d8b274e2-095b-4c14-85f8-2f950fa685c4","Type":"ContainerStarted","Data":"ec4e56e3df78b3601d39ed377fc46fdb23c1a7880414a6250337dc01fbde9a68"}
Sep 10 14:37:09.516897 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:09.516839    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7599-b3cff5" podStartSLOduration=1.516824915 podStartE2EDuration="1.516824915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:09.515160556 +0000 UTC m=+1511.603618105" watchObservedRunningTime="2026-09-10 14:37:09.516824915 +0000 UTC m=+1511.605282485"
Sep 10 14:37:10.095459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:10.095422    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7599-b3cff5"]
Sep 10 14:37:10.101411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:10.101383    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7599-b3cff5"]
Sep 10 14:37:10.506120 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:37:10.506093    2576 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7599-b3cff5" container="test-curl-7599-b3cff5"
Sep 10 14:37:10.642974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:10.642936    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:11.511461 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:11.511425    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7599-b3cff5" podUID="d8b274e2-095b-4c14-85f8-2f950fa685c4" containerName="test-curl-7599-b3cff5" containerID="cri-o://7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d" gracePeriod=2
Sep 10 14:37:11.658545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:11.658517    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:11.690426 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:11.690399    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xszlv\" (UniqueName: \"kubernetes.io/projected/b2b4a96e-425b-4f01-9406-581823b96030-kube-api-access-xszlv\") pod \"b2b4a96e-425b-4f01-9406-581823b96030\" (UID: \"b2b4a96e-425b-4f01-9406-581823b96030\") "
Sep 10 14:37:11.692564 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:11.692541    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2b4a96e-425b-4f01-9406-581823b96030-kube-api-access-xszlv" (OuterVolumeSpecName: "kube-api-access-xszlv") pod "b2b4a96e-425b-4f01-9406-581823b96030" (UID: "b2b4a96e-425b-4f01-9406-581823b96030"). InnerVolumeSpecName "kube-api-access-xszlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:37:11.791386 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:11.791294    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xszlv\" (UniqueName: \"kubernetes.io/projected/b2b4a96e-425b-4f01-9406-581823b96030-kube-api-access-xszlv\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:12.064374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.064284    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"]
Sep 10 14:37:12.082796 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.082756    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:12.428221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.428186    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b2b4a96e-425b-4f01-9406-581823b96030" path="/var/lib/kubelet/pods/b2b4a96e-425b-4f01-9406-581823b96030/volumes"
Sep 10 14:37:12.517762 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.517730    2576 generic.go:356] "Generic (PLEG): container finished" podID="b2b4a96e-425b-4f01-9406-581823b96030" containerID="203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222" exitCode=137
Sep 10 14:37:12.517762 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.517776    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-176b2a"
Sep 10 14:37:12.518225 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.517832    2576 scope.go:117] "RemoveContainer" containerID="203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222"
Sep 10 14:37:12.526366 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.526346    2576 scope.go:117] "RemoveContainer" containerID="203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222"
Sep 10 14:37:12.526639 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:37:12.526621    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222\": container with ID starting with 203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222 not found: ID does not exist" containerID="203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222"
Sep 10 14:37:12.526681 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:12.526646    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222"} err="failed to get container status \"203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222\": rpc error: code = NotFound desc = could not find container \"203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222\": container with ID starting with 203a69a01414d450826ed5f204299feddf13c16c44508bb5d7133cd26e906222 not found: ID does not exist"
Sep 10 14:37:13.213704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:13.213665    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"]
Sep 10 14:37:13.233570 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:13.233537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:13.687614 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:13.687589    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:13.812139 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:13.812041    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w8z7\" (UniqueName: \"kubernetes.io/projected/d8b274e2-095b-4c14-85f8-2f950fa685c4-kube-api-access-7w8z7\") pod \"d8b274e2-095b-4c14-85f8-2f950fa685c4\" (UID: \"d8b274e2-095b-4c14-85f8-2f950fa685c4\") "
Sep 10 14:37:13.814374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:13.814340    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8b274e2-095b-4c14-85f8-2f950fa685c4-kube-api-access-7w8z7" (OuterVolumeSpecName: "kube-api-access-7w8z7") pod "d8b274e2-095b-4c14-85f8-2f950fa685c4" (UID: "d8b274e2-095b-4c14-85f8-2f950fa685c4"). InnerVolumeSpecName "kube-api-access-7w8z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:37:13.913049 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:13.913004    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w8z7\" (UniqueName: \"kubernetes.io/projected/d8b274e2-095b-4c14-85f8-2f950fa685c4-kube-api-access-7w8z7\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:14.428716 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:14.428679    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8b274e2-095b-4c14-85f8-2f950fa685c4" path="/var/lib/kubelet/pods/d8b274e2-095b-4c14-85f8-2f950fa685c4/volumes"
Sep 10 14:37:14.533155 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:14.533119    2576 generic.go:356] "Generic (PLEG): container finished" podID="d8b274e2-095b-4c14-85f8-2f950fa685c4" containerID="7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d" exitCode=137
Sep 10 14:37:14.533342 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:14.533162    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7599-b3cff5"
Sep 10 14:37:14.533342 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:14.533200    2576 scope.go:117] "RemoveContainer" containerID="7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d"
Sep 10 14:37:14.541578 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:14.541560    2576 scope.go:117] "RemoveContainer" containerID="7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d"
Sep 10 14:37:14.541874 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:37:14.541852    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d\": container with ID starting with 7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d not found: ID does not exist" containerID="7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d"
Sep 10 14:37:14.541943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:14.541888    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d"} err="failed to get container status \"7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d\": rpc error: code = NotFound desc = could not find container \"7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d\": container with ID starting with 7277aa3e187c3ef9ad41249855b843c6e5bed8a332fe925bc9b7e5dba1d2b15d not found: ID does not exist"
Sep 10 14:37:15.816245 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:15.816205    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:37:24.396450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:24.396358    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:27.084905 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.084840    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn" podUID="549684a6-e39f-4f1f-99ed-46cddcf8e794" containerName="main" containerID="cri-o://12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559" gracePeriod=45
Sep 10 14:37:27.325031 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.325002    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:27.342065 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.342010    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:37:27.459894 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.459857    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-model-cache\") pod \"549684a6-e39f-4f1f-99ed-46cddcf8e794\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") "
Sep 10 14:37:27.460092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.459923    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/549684a6-e39f-4f1f-99ed-46cddcf8e794-tls-certs\") pod \"549684a6-e39f-4f1f-99ed-46cddcf8e794\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") "
Sep 10 14:37:27.460092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.459967    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-home\") pod \"549684a6-e39f-4f1f-99ed-46cddcf8e794\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") "
Sep 10 14:37:27.460092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460005    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-tmp-dir\") pod \"549684a6-e39f-4f1f-99ed-46cddcf8e794\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") "
Sep 10 14:37:27.460092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460072    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cndtx\" (UniqueName: \"kubernetes.io/projected/549684a6-e39f-4f1f-99ed-46cddcf8e794-kube-api-access-cndtx\") pod \"549684a6-e39f-4f1f-99ed-46cddcf8e794\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") "
Sep 10 14:37:27.460339 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460101    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-dshm\") pod \"549684a6-e39f-4f1f-99ed-46cddcf8e794\" (UID: \"549684a6-e39f-4f1f-99ed-46cddcf8e794\") "
Sep 10 14:37:27.460339 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460144    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-model-cache" (OuterVolumeSpecName: "model-cache") pod "549684a6-e39f-4f1f-99ed-46cddcf8e794" (UID: "549684a6-e39f-4f1f-99ed-46cddcf8e794"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:27.460339 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460297    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-home" (OuterVolumeSpecName: "home") pod "549684a6-e39f-4f1f-99ed-46cddcf8e794" (UID: "549684a6-e39f-4f1f-99ed-46cddcf8e794"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:27.460474 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460359    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "549684a6-e39f-4f1f-99ed-46cddcf8e794" (UID: "549684a6-e39f-4f1f-99ed-46cddcf8e794"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:27.460539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460485    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:27.460539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460503    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:27.460539 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.460512    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:27.462453 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.462422    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/549684a6-e39f-4f1f-99ed-46cddcf8e794-kube-api-access-cndtx" (OuterVolumeSpecName: "kube-api-access-cndtx") pod "549684a6-e39f-4f1f-99ed-46cddcf8e794" (UID: "549684a6-e39f-4f1f-99ed-46cddcf8e794"). InnerVolumeSpecName "kube-api-access-cndtx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:37:27.462591 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.462447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/549684a6-e39f-4f1f-99ed-46cddcf8e794-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "549684a6-e39f-4f1f-99ed-46cddcf8e794" (UID: "549684a6-e39f-4f1f-99ed-46cddcf8e794"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:37:27.462591 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.462447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-dshm" (OuterVolumeSpecName: "dshm") pod "549684a6-e39f-4f1f-99ed-46cddcf8e794" (UID: "549684a6-e39f-4f1f-99ed-46cddcf8e794"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:27.561385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.561343    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/549684a6-e39f-4f1f-99ed-46cddcf8e794-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:27.561385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.561377    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cndtx\" (UniqueName: \"kubernetes.io/projected/549684a6-e39f-4f1f-99ed-46cddcf8e794-kube-api-access-cndtx\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:27.561385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.561386    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/549684a6-e39f-4f1f-99ed-46cddcf8e794-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:27.613448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.613358    2576 generic.go:356] "Generic (PLEG): container finished" podID="6a658b53-8af1-4a91-a52f-630d979de032" containerID="72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54" exitCode=0
Sep 10 14:37:27.613448 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.613438    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" event={"ID":"6a658b53-8af1-4a91-a52f-630d979de032","Type":"ContainerDied","Data":"72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"}
Sep 10 14:37:27.613922 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.613904    2576 scope.go:117] "RemoveContainer" containerID="72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"
Sep 10 14:37:27.614820 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.614796    2576 generic.go:356] "Generic (PLEG): container finished" podID="549684a6-e39f-4f1f-99ed-46cddcf8e794" containerID="12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559" exitCode=0
Sep 10 14:37:27.614920 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.614859    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn" event={"ID":"549684a6-e39f-4f1f-99ed-46cddcf8e794","Type":"ContainerDied","Data":"12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559"}
Sep 10 14:37:27.614920 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.614882    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn" event={"ID":"549684a6-e39f-4f1f-99ed-46cddcf8e794","Type":"ContainerDied","Data":"ac6872785e1b37d51d3166d48b6b6fa5179e03505a328c09bdf1361a3cae8395"}
Sep 10 14:37:27.614920 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.614906    2576 scope.go:117] "RemoveContainer" containerID="12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559"
Sep 10 14:37:27.614920 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.614913    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"
Sep 10 14:37:27.615280 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.615216    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:37:27.615401 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.615355    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:37:27.615470 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.615415    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:37:27.624146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.624122    2576 scope.go:117] "RemoveContainer" containerID="12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559"
Sep 10 14:37:27.624478 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:37:27.624459    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559\": container with ID starting with 12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559 not found: ID does not exist" containerID="12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559"
Sep 10 14:37:27.624541 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.624486    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559"} err="failed to get container status \"12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559\": rpc error: code = NotFound desc = could not find container \"12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559\": container with ID starting with 12030dd815ad1d6a0b6fcf3069bd8e2b7e3720259614c33d700602394fba9559 not found: ID does not exist"
Sep 10 14:37:27.648561 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.648539    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"]
Sep 10 14:37:27.653092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:27.653065    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/sub-test-model-586826-kserve-5d9b9c97d5-h68hn"]
Sep 10 14:37:28.234773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.234699    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq" podUID="bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" containerName="main" containerID="cri-o://27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e" gracePeriod=45
Sep 10 14:37:28.429913 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.429879    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="549684a6-e39f-4f1f-99ed-46cddcf8e794" path="/var/lib/kubelet/pods/549684a6-e39f-4f1f-99ed-46cddcf8e794/volumes"
Sep 10 14:37:28.536754 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.536729    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:37:28.621347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.621314    2576 generic.go:356] "Generic (PLEG): container finished" podID="bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" containerID="27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e" exitCode=0
Sep 10 14:37:28.621518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.621355    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"
Sep 10 14:37:28.621518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.621404    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq" event={"ID":"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a","Type":"ContainerDied","Data":"27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e"}
Sep 10 14:37:28.621518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.621448    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq" event={"ID":"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a","Type":"ContainerDied","Data":"f1ed01f11f328c02058e8262e88128949fbe82a51dcc1f2c2b9af1568e16d107"}
Sep 10 14:37:28.621518 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.621467    2576 scope.go:117] "RemoveContainer" containerID="27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e"
Sep 10 14:37:28.623540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.623517    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" event={"ID":"6a658b53-8af1-4a91-a52f-630d979de032","Type":"ContainerStarted","Data":"4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5"}
Sep 10 14:37:28.630127 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.630111    2576 scope.go:117] "RemoveContainer" containerID="27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e"
Sep 10 14:37:28.630416 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:37:28.630394    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e\": container with ID starting with 27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e not found: ID does not exist" containerID="27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e"
Sep 10 14:37:28.630492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.630429    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e"} err="failed to get container status \"27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e\": rpc error: code = NotFound desc = could not find container \"27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e\": container with ID starting with 27492a099bdabb58b28816087b0accd8632118424b4f0300d3cbda82f4319e8e not found: ID does not exist"
Sep 10 14:37:28.638384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.638362    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:28.639441 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.639414    2576 patch_prober.go:28] interesting pod/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 14:37:28.639510 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.639457    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 14:37:28.686676 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686644    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-model-cache\") pod \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") "
Sep 10 14:37:28.686893 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686725    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-home\") pod \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") "
Sep 10 14:37:28.686893 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686772    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tmp-dir\") pod \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") "
Sep 10 14:37:28.686893 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686793    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-dshm\") pod \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") "
Sep 10 14:37:28.686893 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxjql\" (UniqueName: \"kubernetes.io/projected/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-kube-api-access-pxjql\") pod \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") "
Sep 10 14:37:28.687131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686920    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tls-certs\") pod \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\" (UID: \"bd6aa7ae-a5ff-440c-a56f-0af585a7e47a\") "
Sep 10 14:37:28.687131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.686986    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-model-cache" (OuterVolumeSpecName: "model-cache") pod "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" (UID: "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:28.687131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.687002    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-home" (OuterVolumeSpecName: "home") pod "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" (UID: "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:28.687131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.687015    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" (UID: "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:28.687394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.687371    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:28.687450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.687397    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:28.687450 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.687411    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:28.689659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.689633    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-dshm" (OuterVolumeSpecName: "dshm") pod "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" (UID: "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:37:28.689796 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.689753    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-kube-api-access-pxjql" (OuterVolumeSpecName: "kube-api-access-pxjql") pod "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" (UID: "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a"). InnerVolumeSpecName "kube-api-access-pxjql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:37:28.689860 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.689831    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" (UID: "bd6aa7ae-a5ff-440c-a56f-0af585a7e47a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:37:28.788264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.788127    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:28.788264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.788153    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxjql\" (UniqueName: \"kubernetes.io/projected/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-kube-api-access-pxjql\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:28.788264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.788164    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:37:28.942538 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.942505    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"]
Sep 10 14:37:28.945941 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:28.945915    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-9dcc39/sub-test-model-9dcc39-kserve-5fbc74694f-sv8wq"]
Sep 10 14:37:29.639566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:29.639531    2576 patch_prober.go:28] interesting pod/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 14:37:29.640021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:29.639583    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 14:37:30.246217 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:30.246180    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:30.427366 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:30.427337    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" path="/var/lib/kubelet/pods/bd6aa7ae-a5ff-440c-a56f-0af585a7e47a/volumes"
Sep 10 14:37:30.639351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:30.639267    2576 patch_prober.go:28] interesting pod/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 14:37:30.639351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:30.639318    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 14:37:31.638606 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:31.638575    2576 patch_prober.go:28] interesting pod/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 14:37:31.638982 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:31.638647    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 14:37:32.463011 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:32.462966    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:32.639504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:32.639466    2576 patch_prober.go:28] interesting pod/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Sep 10 14:37:32.639869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:32.639535    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Sep 10 14:37:33.642580 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:33.642546    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:33.643455 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:33.643437    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:37:35.037632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:35.037576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:38.309582 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.309546    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"]
Sep 10 14:37:38.310047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.309980    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8b274e2-095b-4c14-85f8-2f950fa685c4" containerName="test-curl-7599-b3cff5"
Sep 10 14:37:38.310047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.309994    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8b274e2-095b-4c14-85f8-2f950fa685c4" containerName="test-curl-7599-b3cff5"
Sep 10 14:37:38.310047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310012    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="549684a6-e39f-4f1f-99ed-46cddcf8e794" containerName="main"
Sep 10 14:37:38.310047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310018    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="549684a6-e39f-4f1f-99ed-46cddcf8e794" containerName="main"
Sep 10 14:37:38.310047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310025    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" containerName="main"
Sep 10 14:37:38.310047 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310031    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" containerName="main"
Sep 10 14:37:38.310336 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310097    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="549684a6-e39f-4f1f-99ed-46cddcf8e794" containerName="main"
Sep 10 14:37:38.310336 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310106    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8b274e2-095b-4c14-85f8-2f950fa685c4" containerName="test-curl-7599-b3cff5"
Sep 10 14:37:38.310336 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.310114    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd6aa7ae-a5ff-440c-a56f-0af585a7e47a" containerName="main"
Sep 10 14:37:38.314744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.314720    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.318296 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.318271    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"openshift-service-ca.crt\""
Sep 10 14:37:38.318797 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.318668    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"kube-root-ca.crt\""
Sep 10 14:37:38.318950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.318896    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"default-dockercfg-vp8v5\""
Sep 10 14:37:38.318950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.318908    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-586826\"/\"rate-test-model-586826-kserve-self-signed-certs\""
Sep 10 14:37:38.321908 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.321881    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"]
Sep 10 14:37:38.338020 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.337995    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:38.386502 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.386472    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzbw8\" (UniqueName: \"kubernetes.io/projected/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-kube-api-access-mzbw8\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.386659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.386516    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-dshm\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.386659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.386543    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-model-cache\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.386659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.386581    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-home\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.386659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.386630    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tmp-dir\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.386801 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.386661    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tls-certs\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.487795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.487760    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tmp-dir\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.487990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.487811    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tls-certs\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.487990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.487968    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzbw8\" (UniqueName: \"kubernetes.io/projected/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-kube-api-access-mzbw8\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.488147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.488007    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-dshm\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.488147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.488039    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-model-cache\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.488147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.488070    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-home\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.488147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.488127    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tmp-dir\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.488434 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.488406    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-home\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.488434 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.488428    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-model-cache\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.490335 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.490314    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-dshm\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.490484 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.490466    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tls-certs\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.497604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.497575    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzbw8\" (UniqueName: \"kubernetes.io/projected/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-kube-api-access-mzbw8\") pod \"rate-test-model-586826-kserve-6847644967-lrlz8\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") " pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.627192 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.627103    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:38.899670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:38.899646    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"]
Sep 10 14:37:38.901497 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:37:38.901469    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ea54b6d_6a10_4e92_9ae7_e52c7862c539.slice/crio-fdf48903fefa949b26c2f917d1922ed17a69c5df5ffae887860053b227e144f8 WatchSource:0}: Error finding container fdf48903fefa949b26c2f917d1922ed17a69c5df5ffae887860053b227e144f8: Status 404 returned error can't find the container with id fdf48903fefa949b26c2f917d1922ed17a69c5df5ffae887860053b227e144f8
Sep 10 14:37:39.695986 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:39.695947    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8" event={"ID":"2ea54b6d-6a10-4e92-9ae7-e52c7862c539","Type":"ContainerStarted","Data":"80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac"}
Sep 10 14:37:39.696443 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:39.695994    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8" event={"ID":"2ea54b6d-6a10-4e92-9ae7-e52c7862c539","Type":"ContainerStarted","Data":"fdf48903fefa949b26c2f917d1922ed17a69c5df5ffae887860053b227e144f8"}
Sep 10 14:37:39.715880 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:39.715826    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8" podStartSLOduration=1.7158111470000001 podStartE2EDuration="1.715811147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:39.713412735 +0000 UTC m=+1541.801870306" watchObservedRunningTime="2026-09-10 14:37:39.715811147 +0000 UTC m=+1541.804268718"
Sep 10 14:37:40.075511 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:40.075421    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:43.317297 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:43.317265    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:37:46.402087 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:46.402041    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:46.529132 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:46.529091    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:48.627292 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:48.627225    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:48.627292 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:48.627299    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:48.639692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:48.639670    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:48.753599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:48.753556    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:37:52.076446 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:52.076413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:52.177750 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:52.177707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:52.348294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:52.348117    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:52.840677 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:52.840626    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:58.612915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:37:58.612875    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:38:08.100398 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:08.100362    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:08.171959 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:08.171924    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:08.236219 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:08.236177    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:08.301939 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:08.301908    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:08.643805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:08.643773    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:11.507682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:11.507646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:14.257035 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:14.256997    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:14.318909 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:14.318876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:14.493374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:14.493343    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"]
Sep 10 14:38:14.493585 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:14.493558    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" containerID="cri-o://4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5" gracePeriod=30
Sep 10 14:38:14.516716 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:14.516639    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:14.912007 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:14.911931    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:38:15.066925 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:15.066895    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:18.645052 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:18.645013    2576 patch_prober.go:28] interesting pod/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 14:38:18.645454 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:18.645084    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:38:19.745032 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.745009    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:38:19.768901 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.768760    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-istio-token\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.768901 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.768835    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-envoy\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.769367 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.769214    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-certs\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.769834    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bsrdx\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-kube-api-access-bsrdx\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.769905    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-socket\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.769721    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.769938    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-data\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.769972    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a658b53-8af1-4a91-a52f-630d979de032-istiod-ca-cert\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770002    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-credential-socket\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770039    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a658b53-8af1-4a91-a52f-630d979de032-istio-podinfo\") pod \"6a658b53-8af1-4a91-a52f-630d979de032\" (UID: \"6a658b53-8af1-4a91-a52f-630d979de032\") "
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770372    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770464    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770554    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770677    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:38:19.770773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.770712    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a658b53-8af1-4a91-a52f-630d979de032-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:38:19.771816 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.771517    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:38:19.772198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.772167    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:38:19.772789 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.772757    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-kube-api-access-bsrdx" (OuterVolumeSpecName: "kube-api-access-bsrdx") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "kube-api-access-bsrdx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:38:19.772874 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.772802    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a658b53-8af1-4a91-a52f-630d979de032-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a658b53-8af1-4a91-a52f-630d979de032" (UID: "6a658b53-8af1-4a91-a52f-630d979de032"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:38:19.871208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871175    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bsrdx\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-kube-api-access-bsrdx\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871201    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871208 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871210    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871223    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a658b53-8af1-4a91-a52f-630d979de032-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871256    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871264    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a658b53-8af1-4a91-a52f-630d979de032-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871272    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a658b53-8af1-4a91-a52f-630d979de032-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.871487 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.871279    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a658b53-8af1-4a91-a52f-630d979de032-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:19.905985 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.905957    2576 generic.go:356] "Generic (PLEG): container finished" podID="6a658b53-8af1-4a91-a52f-630d979de032" containerID="4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5" exitCode=0
Sep 10 14:38:19.906138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.906031    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"
Sep 10 14:38:19.906138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.906077    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" event={"ID":"6a658b53-8af1-4a91-a52f-630d979de032","Type":"ContainerDied","Data":"4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5"}
Sep 10 14:38:19.906259 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.906151    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr" event={"ID":"6a658b53-8af1-4a91-a52f-630d979de032","Type":"ContainerDied","Data":"77f539ac5292514026c781063f48ad700ee282377fbd4da4cfb4a33ccdf450b7"}
Sep 10 14:38:19.906259 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.906186    2576 scope.go:117] "RemoveContainer" containerID="4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5"
Sep 10 14:38:19.915879 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.915861    2576 scope.go:117] "RemoveContainer" containerID="72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"
Sep 10 14:38:19.924112 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.924092    2576 scope.go:117] "RemoveContainer" containerID="4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5"
Sep 10 14:38:19.924462 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:38:19.924431    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5\": container with ID starting with 4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5 not found: ID does not exist" containerID="4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5"
Sep 10 14:38:19.924527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.924476    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5"} err="failed to get container status \"4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5\": rpc error: code = NotFound desc = could not find container \"4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5\": container with ID starting with 4d6db85f31470761753d7c6321d0a249d6683ce9ef40c5c5e51e25a6fd0bb4c5 not found: ID does not exist"
Sep 10 14:38:19.924527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.924498    2576 scope.go:117] "RemoveContainer" containerID="72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"
Sep 10 14:38:19.924789 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:38:19.924773    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54\": container with ID starting with 72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54 not found: ID does not exist" containerID="72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"
Sep 10 14:38:19.924846 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.924794    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54"} err="failed to get container status \"72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54\": rpc error: code = NotFound desc = could not find container \"72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54\": container with ID starting with 72e79bd7e2e5203969c5519b8649a7e5e5db26a4d628368cfd861187490f6f54 not found: ID does not exist"
Sep 10 14:38:19.930135 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.930112    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"]
Sep 10 14:38:19.935882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:19.935857    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7ae247-openshift-default-5c8db48c46-nq5kr"]
Sep 10 14:38:20.427339 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:20.427304    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a658b53-8af1-4a91-a52f-630d979de032" path="/var/lib/kubelet/pods/6a658b53-8af1-4a91-a52f-630d979de032/volumes"
Sep 10 14:38:25.396927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.396879    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"]
Sep 10 14:38:25.397434 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.397145    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" podUID="b551fd97-8eeb-45e7-b213-764d24674072" containerName="maas-api" containerID="cri-o://e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84" gracePeriod=30
Sep 10 14:38:25.436609 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.436579    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:25.535772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.535737    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:25.609606 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.609576    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:25.651206 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.651122    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:38:25.674671 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.674606    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:25.724856 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.724817    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-metrics-certs\") pod \"b551fd97-8eeb-45e7-b213-764d24674072\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") "
Sep 10 14:38:25.725051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.724865    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2zxvv\" (UniqueName: \"kubernetes.io/projected/b551fd97-8eeb-45e7-b213-764d24674072-kube-api-access-2zxvv\") pod \"b551fd97-8eeb-45e7-b213-764d24674072\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") "
Sep 10 14:38:25.725051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.724914    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls\") pod \"b551fd97-8eeb-45e7-b213-764d24674072\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") "
Sep 10 14:38:25.725051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.725007    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551fd97-8eeb-45e7-b213-764d24674072-openshift-service-ca\") pod \"b551fd97-8eeb-45e7-b213-764d24674072\" (UID: \"b551fd97-8eeb-45e7-b213-764d24674072\") "
Sep 10 14:38:25.725655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.725619    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b551fd97-8eeb-45e7-b213-764d24674072-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b551fd97-8eeb-45e7-b213-764d24674072" (UID: "b551fd97-8eeb-45e7-b213-764d24674072"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:38:25.727419 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.727390    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b551fd97-8eeb-45e7-b213-764d24674072-kube-api-access-2zxvv" (OuterVolumeSpecName: "kube-api-access-2zxvv") pod "b551fd97-8eeb-45e7-b213-764d24674072" (UID: "b551fd97-8eeb-45e7-b213-764d24674072"). InnerVolumeSpecName "kube-api-access-2zxvv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:38:25.727521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.727428    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b551fd97-8eeb-45e7-b213-764d24674072" (UID: "b551fd97-8eeb-45e7-b213-764d24674072"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:38:25.727521 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.727469    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b551fd97-8eeb-45e7-b213-764d24674072" (UID: "b551fd97-8eeb-45e7-b213-764d24674072"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:38:25.826004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.825970    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:25.826004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.826001    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2zxvv\" (UniqueName: \"kubernetes.io/projected/b551fd97-8eeb-45e7-b213-764d24674072-kube-api-access-2zxvv\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:25.826004 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.826013    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b551fd97-8eeb-45e7-b213-764d24674072-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:25.826270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.826021    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b551fd97-8eeb-45e7-b213-764d24674072-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:25.937583 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.937545    2576 generic.go:356] "Generic (PLEG): container finished" podID="b551fd97-8eeb-45e7-b213-764d24674072" containerID="e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84" exitCode=0
Sep 10 14:38:25.937765 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.937595    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"
Sep 10 14:38:25.937765 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.937605    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" event={"ID":"b551fd97-8eeb-45e7-b213-764d24674072","Type":"ContainerDied","Data":"e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84"}
Sep 10 14:38:25.937765 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.937648    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj" event={"ID":"b551fd97-8eeb-45e7-b213-764d24674072","Type":"ContainerDied","Data":"3a5c39dc2a87dbe528196eed4bc7fd713f6e25510ce94386957429f84ab3c8e8"}
Sep 10 14:38:25.937765 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.937668    2576 scope.go:117] "RemoveContainer" containerID="e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84"
Sep 10 14:38:25.947377 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.947359    2576 scope.go:117] "RemoveContainer" containerID="e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84"
Sep 10 14:38:25.947697 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:38:25.947673    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84\": container with ID starting with e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84 not found: ID does not exist" containerID="e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84"
Sep 10 14:38:25.947748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.947709    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84"} err="failed to get container status \"e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84\": rpc error: code = NotFound desc = could not find container \"e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84\": container with ID starting with e1e5479232908bf2c27d27cc0053b0b8a29bf3d6dae69dd3cac0879e6bdd2f84 not found: ID does not exist"
Sep 10 14:38:25.958355 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.958322    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"]
Sep 10 14:38:25.963883 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.963845    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-9e5db6-6664d56bbf-7xlqj"]
Sep 10 14:38:25.965670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:25.965646    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:26.427246 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:26.427211    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b551fd97-8eeb-45e7-b213-764d24674072" path="/var/lib/kubelet/pods/b551fd97-8eeb-45e7-b213-764d24674072/volumes"
Sep 10 14:38:28.714950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:28.714924    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:38:31.803958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:31.803923    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:33.302449 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:33.302413    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:41.814137 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:41.814106    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:41.910424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:41.910387    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:38:42.626599 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:42.626564    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:43.104996 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.104957    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"]
Sep 10 14:38:43.105456 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.105203    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9" podUID="7f16b67f-6fed-42bc-a888-41f6f786ef1f" containerName="maas-api" containerID="cri-o://0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829" gracePeriod=30
Sep 10 14:38:43.132767 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.132741    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:43.233403 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.233374    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:43.299547 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.299513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:43.359901 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.359838    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:38:43.361027 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.361003    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:43.494140 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.494106    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-metrics-certs\") pod \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") "
Sep 10 14:38:43.494330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.494190    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-maas-api-tls\") pod \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") "
Sep 10 14:38:43.494330 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.494323    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16b67f-6fed-42bc-a888-41f6f786ef1f-openshift-service-ca\") pod \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") "
Sep 10 14:38:43.494416 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.494369    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh2bt\" (UniqueName: \"kubernetes.io/projected/7f16b67f-6fed-42bc-a888-41f6f786ef1f-kube-api-access-nh2bt\") pod \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\" (UID: \"7f16b67f-6fed-42bc-a888-41f6f786ef1f\") "
Sep 10 14:38:43.494690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.494658    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f16b67f-6fed-42bc-a888-41f6f786ef1f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f16b67f-6fed-42bc-a888-41f6f786ef1f" (UID: "7f16b67f-6fed-42bc-a888-41f6f786ef1f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:38:43.496602 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.496574    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f16b67f-6fed-42bc-a888-41f6f786ef1f-kube-api-access-nh2bt" (OuterVolumeSpecName: "kube-api-access-nh2bt") pod "7f16b67f-6fed-42bc-a888-41f6f786ef1f" (UID: "7f16b67f-6fed-42bc-a888-41f6f786ef1f"). InnerVolumeSpecName "kube-api-access-nh2bt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:38:43.496931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.496909    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f16b67f-6fed-42bc-a888-41f6f786ef1f" (UID: "7f16b67f-6fed-42bc-a888-41f6f786ef1f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:38:43.496984 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.496926    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f16b67f-6fed-42bc-a888-41f6f786ef1f" (UID: "7f16b67f-6fed-42bc-a888-41f6f786ef1f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:38:43.595613 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.595576    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:43.595613 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.595608    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f16b67f-6fed-42bc-a888-41f6f786ef1f-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:43.595613 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.595620    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh2bt\" (UniqueName: \"kubernetes.io/projected/7f16b67f-6fed-42bc-a888-41f6f786ef1f-kube-api-access-nh2bt\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:43.595836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.595628    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f16b67f-6fed-42bc-a888-41f6f786ef1f-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:38:43.702796 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:43.702763    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:44.034805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.034717    2576 generic.go:356] "Generic (PLEG): container finished" podID="7f16b67f-6fed-42bc-a888-41f6f786ef1f" containerID="0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829" exitCode=0
Sep 10 14:38:44.034805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.034758    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"
Sep 10 14:38:44.034995 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.034814    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9" event={"ID":"7f16b67f-6fed-42bc-a888-41f6f786ef1f","Type":"ContainerDied","Data":"0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829"}
Sep 10 14:38:44.034995 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.034860    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9" event={"ID":"7f16b67f-6fed-42bc-a888-41f6f786ef1f","Type":"ContainerDied","Data":"5a923bb731c0cd76d62f02a94cc627f08055738e98697505969335712f42fb54"}
Sep 10 14:38:44.034995 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.034883    2576 scope.go:117] "RemoveContainer" containerID="0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829"
Sep 10 14:38:44.044830 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.044807    2576 scope.go:117] "RemoveContainer" containerID="0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829"
Sep 10 14:38:44.045106 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:38:44.045087    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829\": container with ID starting with 0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829 not found: ID does not exist" containerID="0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829"
Sep 10 14:38:44.045158 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.045115    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829"} err="failed to get container status \"0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829\": rpc error: code = NotFound desc = could not find container \"0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829\": container with ID starting with 0597dfbc66125fd925305d3979bd4935ba5f1babf749143a1b8301e28701e829 not found: ID does not exist"
Sep 10 14:38:44.056615 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.056589    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"]
Sep 10 14:38:44.063057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.063035    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-54c040-6fd6cf6957-f4tw9"]
Sep 10 14:38:44.426993 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:44.426961    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f16b67f-6fed-42bc-a888-41f6f786ef1f" path="/var/lib/kubelet/pods/7f16b67f-6fed-42bc-a888-41f6f786ef1f/volumes"
Sep 10 14:38:49.336033 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:49.335950    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:49.435504 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:49.435460    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:49.619211 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:49.619113    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:56.016937 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:38:56.016897    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:39:00.370067 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:00.370027    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"]
Sep 10 14:39:00.428528 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:00.428489    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:01.525592 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:01.525557    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:01.860730 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:01.860651    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:02.159452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:02.159365    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:02.419696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:02.419610    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:05.518854 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:05.518817    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:39:05.672207 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:05.672130    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:05.766684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:05.766653    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:06.850836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.850801    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"]
Sep 10 14:39:06.851202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851182    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b551fd97-8eeb-45e7-b213-764d24674072" containerName="maas-api"
Sep 10 14:39:06.851202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851199    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="b551fd97-8eeb-45e7-b213-764d24674072" containerName="maas-api"
Sep 10 14:39:06.851302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851226    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f16b67f-6fed-42bc-a888-41f6f786ef1f" containerName="maas-api"
Sep 10 14:39:06.851302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851247    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f16b67f-6fed-42bc-a888-41f6f786ef1f" containerName="maas-api"
Sep 10 14:39:06.851302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851253    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy"
Sep 10 14:39:06.851302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851258    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy"
Sep 10 14:39:06.851302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851270    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy"
Sep 10 14:39:06.851302 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851276    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy"
Sep 10 14:39:06.851482 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851345    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f16b67f-6fed-42bc-a888-41f6f786ef1f" containerName="maas-api"
Sep 10 14:39:06.851482 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851358    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="b551fd97-8eeb-45e7-b213-764d24674072" containerName="maas-api"
Sep 10 14:39:06.851482 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851365    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy"
Sep 10 14:39:06.851482 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.851375    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a658b53-8af1-4a91-a52f-630d979de032" containerName="istio-proxy"
Sep 10 14:39:06.856065 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.856045    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:06.861737 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.861708    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-90366e-serving-cert\""
Sep 10 14:39:06.862066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.862050    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:39:06.863088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.862726    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:39:06.863088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.862762    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:39:06.863088 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.862812    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:39:06.868527 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.868505    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"]
Sep 10 14:39:06.903776 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.903749    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:06.903966 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.903807    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-metrics-certs\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:06.903966 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.903844    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18778b8d-2cec-4f05-b2a2-69ff76afc41f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:06.903966 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.903929    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5x2mc\" (UniqueName: \"kubernetes.io/projected/18778b8d-2cec-4f05-b2a2-69ff76afc41f-kube-api-access-5x2mc\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:06.957034 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:06.957004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:07.004806 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.004770    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5x2mc\" (UniqueName: \"kubernetes.io/projected/18778b8d-2cec-4f05-b2a2-69ff76afc41f-kube-api-access-5x2mc\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.004806 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.004808    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.005050 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.004840    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-metrics-certs\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.005050 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.004865    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18778b8d-2cec-4f05-b2a2-69ff76afc41f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.005601 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.005531    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18778b8d-2cec-4f05-b2a2-69ff76afc41f-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.007490 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.007460    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-metrics-certs\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.007594 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.007496    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-maas-api-tls\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.014951 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.014929    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5x2mc\" (UniqueName: \"kubernetes.io/projected/18778b8d-2cec-4f05-b2a2-69ff76afc41f-kube-api-access-5x2mc\") pod \"maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.053771 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.053742    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:07.166618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.166536    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:07.439943 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.439915    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"]
Sep 10 14:39:07.443322 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:39:07.443281    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18778b8d_2cec_4f05_b2a2_69ff76afc41f.slice/crio-4b0d643856d30044e66ff00a4e2f14d82f70d328f7490d21d5c283e10c4a33e5 WatchSource:0}: Error finding container 4b0d643856d30044e66ff00a4e2f14d82f70d328f7490d21d5c283e10c4a33e5: Status 404 returned error can't find the container with id 4b0d643856d30044e66ff00a4e2f14d82f70d328f7490d21d5c283e10c4a33e5
Sep 10 14:39:07.444740 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.444721    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:39:07.504547 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.504504    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:07.598496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:07.598461    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:08.167660 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:08.167626    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr" event={"ID":"18778b8d-2cec-4f05-b2a2-69ff76afc41f","Type":"ContainerStarted","Data":"dc719231f6b33c090518547fe0bc3c5077c2d3925978c65731ee2ed4cd1d3b43"}
Sep 10 14:39:08.168124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:08.167684    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr" event={"ID":"18778b8d-2cec-4f05-b2a2-69ff76afc41f","Type":"ContainerStarted","Data":"4b0d643856d30044e66ff00a4e2f14d82f70d328f7490d21d5c283e10c4a33e5"}
Sep 10 14:39:08.168124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:08.167790    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:08.187083 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:08.187038    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr" podStartSLOduration=1.657773436 podStartE2EDuration="2.187022286s" totalImagesPullingTime="529.24885ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:08.185172438 +0000 UTC m=+1630.273630010" watchObservedRunningTime="2026-09-10 14:39:08.187022286 +0000 UTC m=+1630.275479857"
Sep 10 14:39:14.178988 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:14.178958    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:39:15.389395 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.389354    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8" podUID="2ea54b6d-6a10-4e92-9ae7-e52c7862c539" containerName="main" containerID="cri-o://80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac" gracePeriod=45
Sep 10 14:39:15.635772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.635749    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:39:15.680143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tls-certs\") pod \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") "
Sep 10 14:39:15.680346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680172    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-home\") pod \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") "
Sep 10 14:39:15.680346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680199    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-model-cache\") pod \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") "
Sep 10 14:39:15.680346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680256    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-dshm\") pod \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") "
Sep 10 14:39:15.680346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680285    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tmp-dir\") pod \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") "
Sep 10 14:39:15.680346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680310    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzbw8\" (UniqueName: \"kubernetes.io/projected/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-kube-api-access-mzbw8\") pod \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\" (UID: \"2ea54b6d-6a10-4e92-9ae7-e52c7862c539\") "
Sep 10 14:39:15.680629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680509    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-home" (OuterVolumeSpecName: "home") pod "2ea54b6d-6a10-4e92-9ae7-e52c7862c539" (UID: "2ea54b6d-6a10-4e92-9ae7-e52c7862c539"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:15.680629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680526    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-model-cache" (OuterVolumeSpecName: "model-cache") pod "2ea54b6d-6a10-4e92-9ae7-e52c7862c539" (UID: "2ea54b6d-6a10-4e92-9ae7-e52c7862c539"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:15.680629 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680600    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2ea54b6d-6a10-4e92-9ae7-e52c7862c539" (UID: "2ea54b6d-6a10-4e92-9ae7-e52c7862c539"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:15.680773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680676    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:39:15.680773 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.680696    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:39:15.682543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.682520    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-dshm" (OuterVolumeSpecName: "dshm") pod "2ea54b6d-6a10-4e92-9ae7-e52c7862c539" (UID: "2ea54b6d-6a10-4e92-9ae7-e52c7862c539"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:39:15.682657 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.682595    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2ea54b6d-6a10-4e92-9ae7-e52c7862c539" (UID: "2ea54b6d-6a10-4e92-9ae7-e52c7862c539"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:39:15.682700 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.682661    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-kube-api-access-mzbw8" (OuterVolumeSpecName: "kube-api-access-mzbw8") pod "2ea54b6d-6a10-4e92-9ae7-e52c7862c539" (UID: "2ea54b6d-6a10-4e92-9ae7-e52c7862c539"). InnerVolumeSpecName "kube-api-access-mzbw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:39:15.781732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.781696    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:39:15.781732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.781728    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:39:15.781732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.781737    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzbw8\" (UniqueName: \"kubernetes.io/projected/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-kube-api-access-mzbw8\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:39:15.781958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:15.781746    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2ea54b6d-6a10-4e92-9ae7-e52c7862c539-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:39:16.107880 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.107793    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:39:16.210455 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.210419    2576 generic.go:356] "Generic (PLEG): container finished" podID="2ea54b6d-6a10-4e92-9ae7-e52c7862c539" containerID="80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac" exitCode=0
Sep 10 14:39:16.210608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.210467    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"
Sep 10 14:39:16.210608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.210504    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8" event={"ID":"2ea54b6d-6a10-4e92-9ae7-e52c7862c539","Type":"ContainerDied","Data":"80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac"}
Sep 10 14:39:16.210608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.210550    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8" event={"ID":"2ea54b6d-6a10-4e92-9ae7-e52c7862c539","Type":"ContainerDied","Data":"fdf48903fefa949b26c2f917d1922ed17a69c5df5ffae887860053b227e144f8"}
Sep 10 14:39:16.210608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.210568    2576 scope.go:117] "RemoveContainer" containerID="80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac"
Sep 10 14:39:16.219424 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.219406    2576 scope.go:117] "RemoveContainer" containerID="80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac"
Sep 10 14:39:16.219693 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:39:16.219674    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac\": container with ID starting with 80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac not found: ID does not exist" containerID="80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac"
Sep 10 14:39:16.219752 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.219702    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac"} err="failed to get container status \"80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac\": rpc error: code = NotFound desc = could not find container \"80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac\": container with ID starting with 80b2e8a17a870b979be97118f9e9afe7a75b8b27ea24678ca9347cfd65d777ac not found: ID does not exist"
Sep 10 14:39:16.231353 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.231330    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"]
Sep 10 14:39:16.235543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.235515    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-586826/rate-test-model-586826-kserve-6847644967-lrlz8"]
Sep 10 14:39:16.427948 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:16.427910    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2ea54b6d-6a10-4e92-9ae7-e52c7862c539" path="/var/lib/kubelet/pods/2ea54b6d-6a10-4e92-9ae7-e52c7862c539/volumes"
Sep 10 14:39:29.661083 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:29.661035    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:29.725777 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:29.725746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:30.619914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:30.619877    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:31.267469 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:31.267428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:31.586828 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:31.586746    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:31.784968 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:31.784933    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:34.917488 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:34.917453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:35.032787 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:35.032757    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:36.007904 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.007828    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"]
Sep 10 14:39:36.008574 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.008552    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2ea54b6d-6a10-4e92-9ae7-e52c7862c539" containerName="main"
Sep 10 14:39:36.008632 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.008582    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2ea54b6d-6a10-4e92-9ae7-e52c7862c539" containerName="main"
Sep 10 14:39:36.008692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.008679    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2ea54b6d-6a10-4e92-9ae7-e52c7862c539" containerName="main"
Sep 10 14:39:36.011655 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.011636    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.014348 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.014323    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-6b9e31-serving-cert\""
Sep 10 14:39:36.019974 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.019950    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"]
Sep 10 14:39:36.048095 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.048054    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:36.145492 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.145457    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:36.165753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.165708    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-metrics-certs\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.165931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.165775    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4zwx\" (UniqueName: \"kubernetes.io/projected/a73722b1-d370-4ec8-85e5-c7053443427e-kube-api-access-t4zwx\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.165931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.165875    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.166039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.165927    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73722b1-d370-4ec8-85e5-c7053443427e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.267091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.266991    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-metrics-certs\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.267091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.267042    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4zwx\" (UniqueName: \"kubernetes.io/projected/a73722b1-d370-4ec8-85e5-c7053443427e-kube-api-access-t4zwx\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.267091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.267079    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.267393 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.267106    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73722b1-d370-4ec8-85e5-c7053443427e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.267393 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:39:36.267261    2576 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-serving-cert: secret "maas-api-e2e-ipp-b-6b9e31-serving-cert" not found
Sep 10 14:39:36.267393 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:39:36.267333    2576 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls podName:a73722b1-d370-4ec8-85e5-c7053443427e nodeName:}" failed. No retries permitted until 2026-09-10 14:39:36.76731273 +0000 UTC m=+1658.855770279 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls") pod "maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" (UID: "a73722b1-d370-4ec8-85e5-c7053443427e") : secret "maas-api-e2e-ipp-b-6b9e31-serving-cert" not found
Sep 10 14:39:36.267835 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.267816    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73722b1-d370-4ec8-85e5-c7053443427e-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.269640 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.269619    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-metrics-certs\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.275756 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.275728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4zwx\" (UniqueName: \"kubernetes.io/projected/a73722b1-d370-4ec8-85e5-c7053443427e-kube-api-access-t4zwx\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.402520 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.402483    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:36.519962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.519875    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:36.613942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.613901    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:36.771753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.771661    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.774333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.774307    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls\") pod \"maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:36.926022 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:36.925982    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:37.201604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:37.201575    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"]
Sep 10 14:39:37.204296 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:39:37.204269    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda73722b1_d370_4ec8_85e5_c7053443427e.slice/crio-8aef8fda202f3d35a508a8e58c530fa281c4032d37ea3ef4da7a075245b89c13 WatchSource:0}: Error finding container 8aef8fda202f3d35a508a8e58c530fa281c4032d37ea3ef4da7a075245b89c13: Status 404 returned error can't find the container with id 8aef8fda202f3d35a508a8e58c530fa281c4032d37ea3ef4da7a075245b89c13
Sep 10 14:39:37.320500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:37.320461    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" event={"ID":"a73722b1-d370-4ec8-85e5-c7053443427e","Type":"ContainerStarted","Data":"8aef8fda202f3d35a508a8e58c530fa281c4032d37ea3ef4da7a075245b89c13"}
Sep 10 14:39:38.327219 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:38.327175    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" event={"ID":"a73722b1-d370-4ec8-85e5-c7053443427e","Type":"ContainerStarted","Data":"b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a"}
Sep 10 14:39:38.327715 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:38.327317    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:38.344553 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:38.344502    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" podStartSLOduration=3.015495651 podStartE2EDuration="3.344487308s" totalImagesPullingTime="328.991657ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:38.342969391 +0000 UTC m=+1660.431426976" watchObservedRunningTime="2026-09-10 14:39:38.344487308 +0000 UTC m=+1660.432944878"
Sep 10 14:39:42.622806 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:42.622768    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:39:44.337736 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:44.337705    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:39:51.667206 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:51.667168    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:59.869228 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:39:59.869185    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:18.599168 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.599128    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"]
Sep 10 14:40:18.604343 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.604320    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.607727 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.607699    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-90366e\"/\"default-dockercfg-x7gw4\""
Sep 10 14:40:18.607872 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.607771    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-90366e\"/\"kube-root-ca.crt\""
Sep 10 14:40:18.608253 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.608205    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-90366e\"/\"openshift-service-ca.crt\""
Sep 10 14:40:18.608407 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.608390    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-90366e\"/\"ipp-route-90366e-kserve-self-signed-certs\""
Sep 10 14:40:18.609669 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.609591    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"]
Sep 10 14:40:18.643294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.643261    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:18.726156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.726117    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d53dda8-9f3c-4dd3-b228-426520874059-tls-certs\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.726156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.726160    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-dshm\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.726429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.726258    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hglvq\" (UniqueName: \"kubernetes.io/projected/2d53dda8-9f3c-4dd3-b228-426520874059-kube-api-access-hglvq\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.726429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.726308    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-home\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.726429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.726351    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-tmp-dir\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.726429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.726383    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-model-cache\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.827322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.827281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d53dda8-9f3c-4dd3-b228-426520874059-tls-certs\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.827322 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.827332    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-dshm\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.827589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.827364    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hglvq\" (UniqueName: \"kubernetes.io/projected/2d53dda8-9f3c-4dd3-b228-426520874059-kube-api-access-hglvq\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.827589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.827390    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-home\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.827953 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.827647    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-tmp-dir\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.828067 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.828005    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-model-cache\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.828297 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.828272    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-tmp-dir\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.828489 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.828313    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-home\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.828749 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.828720    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-model-cache\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.830694 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.830662    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-dshm\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.831063 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.831038    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d53dda8-9f3c-4dd3-b228-426520874059-tls-certs\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.837506 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.837480    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hglvq\" (UniqueName: \"kubernetes.io/projected/2d53dda8-9f3c-4dd3-b228-426520874059-kube-api-access-hglvq\") pod \"ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") " pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:18.917428 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:18.917335    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:19.191449 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:19.191424    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"]
Sep 10 14:40:19.193329 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:40:19.193301    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2d53dda8_9f3c_4dd3_b228_426520874059.slice/crio-dd4af557cd23e1daa9c16fe7aa0d48fe315a9a7d8510c5562a47626ab942381e WatchSource:0}: Error finding container dd4af557cd23e1daa9c16fe7aa0d48fe315a9a7d8510c5562a47626ab942381e: Status 404 returned error can't find the container with id dd4af557cd23e1daa9c16fe7aa0d48fe315a9a7d8510c5562a47626ab942381e
Sep 10 14:40:19.532173 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:19.532083    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c" event={"ID":"2d53dda8-9f3c-4dd3-b228-426520874059","Type":"ContainerStarted","Data":"9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59"}
Sep 10 14:40:19.532173 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:19.532122    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c" event={"ID":"2d53dda8-9f3c-4dd3-b228-426520874059","Type":"ContainerStarted","Data":"dd4af557cd23e1daa9c16fe7aa0d48fe315a9a7d8510c5562a47626ab942381e"}
Sep 10 14:40:19.550926 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:19.550864    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c" podStartSLOduration=1.550843025 podStartE2EDuration="1.550843025s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:19.548076669 +0000 UTC m=+1701.636534260" watchObservedRunningTime="2026-09-10 14:40:19.550843025 +0000 UTC m=+1701.639300598"
Sep 10 14:40:22.616548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:22.616517    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:40:23.230049 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:23.230011    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:28.918201 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:28.918163    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:28.918693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:28.918222    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:28.931420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:28.931393    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:29.597383 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:29.597355    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:40:39.222374 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:39.222340    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:40:46.119009 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.118969    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:46.388680 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.388598    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"]
Sep 10 14:40:46.392170 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.392149    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.394755 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.394736    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-652186-openshift-default-dockercfg-p4knf\""
Sep 10 14:40:46.404024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.403992    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"]
Sep 10 14:40:46.420288 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.420254    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:46.492012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.491978    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492017    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-podinfo\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492057    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-credential-socket\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492099    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-certs\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492130    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-envoy\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492147    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mlvc\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-kube-api-access-4mlvc\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492172    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-data\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492340 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492325    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-socket\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.492733 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.492377    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-token\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593471 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593430    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-token\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593471 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593476    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593508    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-podinfo\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593544    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-credential-socket\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593574    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-certs\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593733    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-envoy\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593781    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mlvc\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-kube-api-access-4mlvc\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.593915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593812    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-data\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.594081 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.593998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-credential-socket\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.594081 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.594001    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-socket\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.594081 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.594051    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-certs\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.594443 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.594265    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-socket\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.594443 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.594303    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-data\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.595096 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.595075    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istiod-ca-cert\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.596388 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.596365    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-envoy\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.596501 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.596482    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-podinfo\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.601631 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.601605    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mlvc\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-kube-api-access-4mlvc\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.601749 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.601641    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-token\") pod \"e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") " pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.705368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.705331    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:46.751563 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.751523    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:46.988633 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.988548    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"]
Sep 10 14:40:46.991782 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:40:46.991748    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d014f0_f76d_4e7d_8fe6_5740126c9deb.slice/crio-7df4c9b86586f8d760418d5e96bd5de92e25384bed1a263a0c2e0995a349e3f1 WatchSource:0}: Error finding container 7df4c9b86586f8d760418d5e96bd5de92e25384bed1a263a0c2e0995a349e3f1: Status 404 returned error can't find the container with id 7df4c9b86586f8d760418d5e96bd5de92e25384bed1a263a0c2e0995a349e3f1
Sep 10 14:40:46.994270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.994213    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:40:46.994344 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.994318    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:40:46.994381 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:46.994357    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:40:47.675978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:47.675946    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" event={"ID":"04d014f0-f76d-4e7d-8fe6-5740126c9deb","Type":"ContainerStarted","Data":"c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"}
Sep 10 14:40:47.676346 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:47.675995    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" event={"ID":"04d014f0-f76d-4e7d-8fe6-5740126c9deb","Type":"ContainerStarted","Data":"7df4c9b86586f8d760418d5e96bd5de92e25384bed1a263a0c2e0995a349e3f1"}
Sep 10 14:40:47.696683 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:47.696640    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podStartSLOduration=1.696625521 podStartE2EDuration="1.696625521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:47.695411492 +0000 UTC m=+1729.783869064" watchObservedRunningTime="2026-09-10 14:40:47.696625521 +0000 UTC m=+1729.785083091"
Sep 10 14:40:47.705524 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:47.705502    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:48.711105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:48.711078    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:49.567466 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:49.567428    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:49.669645 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:49.669599    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:49.686488 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:49.686457    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:49.687341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:49.687324    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:40:50.696069 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:50.696036    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:50.796131 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:50.796099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.052255 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:51.052145    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.203623 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:51.203589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.301682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:51.301650    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.607269 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:40:58.607218    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:41:08.784345 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:08.784313    2576 generic.go:356] "Generic (PLEG): container finished" podID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerID="c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68" exitCode=0
Sep 10 14:41:08.784739 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:08.784387    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" event={"ID":"04d014f0-f76d-4e7d-8fe6-5740126c9deb","Type":"ContainerDied","Data":"c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"}
Sep 10 14:41:08.784837 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:08.784821    2576 scope.go:117] "RemoveContainer" containerID="c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"
Sep 10 14:41:08.786029 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:08.785994    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:41:08.786138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:08.786079    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:41:08.786138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:08.786116    2576 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:41:09.792523 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:09.792487    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" event={"ID":"04d014f0-f76d-4e7d-8fe6-5740126c9deb","Type":"ContainerStarted","Data":"e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210"}
Sep 10 14:41:10.705474 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:10.705436    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:41:10.705767 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:10.705744    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 14:41:10.705867 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:10.705800    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 14:41:11.706370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:11.706334    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 14:41:11.706744 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:11.706405    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 14:41:12.705673 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:12.705636    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 14:41:12.705852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:12.705693    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 14:41:13.706123 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:13.706086    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 14:41:13.706542 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:13.706161    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 14:41:14.508503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:14.508459    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:14.705961 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:14.705921    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Sep 10 14:41:14.706141 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:14.705977    2576 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Sep 10 14:41:15.710349 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:15.710320    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:41:15.710919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:15.710662    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:41:15.711305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:15.711286    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:41:21.396470 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:21.396417    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:21.498198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:21.498162    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:21.596950 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:21.596919    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:21.660966 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:21.660881    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:21.957782 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:21.957751    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:25.818615 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:25.818583    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:41:27.612198 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:27.612159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:27.690559 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:27.690506    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:27.871838 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:27.871763    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"]
Sep 10 14:41:27.872051 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:27.872027    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" containerID="cri-o://e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210" gracePeriod=30
Sep 10 14:41:27.921965 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:27.921928    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:30.713379 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:30.713341    2576 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 10 14:41:30.713751 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:30.713435    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:41:31.551426 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:31.551395    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:33.122615 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.122586    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:41:33.240044 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240010    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-data\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240044 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240043    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-socket\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240062    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-envoy\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240097    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mlvc\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-kube-api-access-4mlvc\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240118    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-podinfo\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240149    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-certs\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240171    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-credential-socket\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240194    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istiod-ca-cert\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240284    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-token\") pod \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\" (UID: \"04d014f0-f76d-4e7d-8fe6-5740126c9deb\") "
Sep 10 14:41:33.240368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240349    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:41:33.240805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240447    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-data" (OuterVolumeSpecName: "istio-data") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:41:33.240805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240639    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.240805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240659    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.240805 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240690    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:41:33.241189 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240827    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:41:33.241189 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.240838    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:33.242550 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.242529    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:41:33.242661 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.242593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:41:33.243017 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.242996    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-kube-api-access-4mlvc" (OuterVolumeSpecName: "kube-api-access-4mlvc") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "kube-api-access-4mlvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:33.243057 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.243017    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-token" (OuterVolumeSpecName: "istio-token") pod "04d014f0-f76d-4e7d-8fe6-5740126c9deb" (UID: "04d014f0-f76d-4e7d-8fe6-5740126c9deb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:33.341652 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341614    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.341652 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341646    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.341652 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341660    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.341876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341671    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.341876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341683    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mlvc\" (UniqueName: \"kubernetes.io/projected/04d014f0-f76d-4e7d-8fe6-5740126c9deb-kube-api-access-4mlvc\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.341876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341700    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04d014f0-f76d-4e7d-8fe6-5740126c9deb-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.341876 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.341711    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04d014f0-f76d-4e7d-8fe6-5740126c9deb-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:33.917411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.917377    2576 generic.go:356] "Generic (PLEG): container finished" podID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerID="e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210" exitCode=0
Sep 10 14:41:33.917618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.917430    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" event={"ID":"04d014f0-f76d-4e7d-8fe6-5740126c9deb","Type":"ContainerDied","Data":"e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210"}
Sep 10 14:41:33.917618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.917454    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"
Sep 10 14:41:33.917618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.917474    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2" event={"ID":"04d014f0-f76d-4e7d-8fe6-5740126c9deb","Type":"ContainerDied","Data":"7df4c9b86586f8d760418d5e96bd5de92e25384bed1a263a0c2e0995a349e3f1"}
Sep 10 14:41:33.917618 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.917494    2576 scope.go:117] "RemoveContainer" containerID="e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210"
Sep 10 14:41:33.927612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.927590    2576 scope.go:117] "RemoveContainer" containerID="c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"
Sep 10 14:41:33.936263 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.936244    2576 scope.go:117] "RemoveContainer" containerID="e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210"
Sep 10 14:41:33.936536 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:41:33.936518    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210\": container with ID starting with e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210 not found: ID does not exist" containerID="e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210"
Sep 10 14:41:33.936584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.936543    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210"} err="failed to get container status \"e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210\": rpc error: code = NotFound desc = could not find container \"e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210\": container with ID starting with e0ecf10a405d0cc296affa907de36525b3ec77be5d1b82c1afd64250f9f25210 not found: ID does not exist"
Sep 10 14:41:33.936584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.936559    2576 scope.go:117] "RemoveContainer" containerID="c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"
Sep 10 14:41:33.936847 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:41:33.936828    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68\": container with ID starting with c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68 not found: ID does not exist" containerID="c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"
Sep 10 14:41:33.936904 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.936858    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68"} err="failed to get container status \"c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68\": rpc error: code = NotFound desc = could not find container \"c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68\": container with ID starting with c787a865afd4bd09cc97aafa41122b0885a6179d6d8184ccca20fd92eaf55a68 not found: ID does not exist"
Sep 10 14:41:33.940052 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.940032    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"]
Sep 10 14:41:33.946108 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:33.946087    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-652186-openshift-default-6f9649df44-sqqs2"]
Sep 10 14:41:34.428413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:34.428377    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" path="/var/lib/kubelet/pods/04d014f0-f76d-4e7d-8fe6-5740126c9deb/volumes"
Sep 10 14:41:36.915635 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:36.915578    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:44.512466 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:44.512428    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:41:46.821363 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:46.821326    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"]
Sep 10 14:41:46.821928 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:46.821553    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr" podUID="18778b8d-2cec-4f05-b2a2-69ff76afc41f" containerName="maas-api" containerID="cri-o://dc719231f6b33c090518547fe0bc3c5077c2d3925978c65731ee2ed4cd1d3b43" gracePeriod=30
Sep 10 14:41:46.860747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:46.860707    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:46.958147 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:46.958114    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:46.986672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:46.986639    2576 generic.go:356] "Generic (PLEG): container finished" podID="18778b8d-2cec-4f05-b2a2-69ff76afc41f" containerID="dc719231f6b33c090518547fe0bc3c5077c2d3925978c65731ee2ed4cd1d3b43" exitCode=0
Sep 10 14:41:46.986847 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:46.986712    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr" event={"ID":"18778b8d-2cec-4f05-b2a2-69ff76afc41f","Type":"ContainerDied","Data":"dc719231f6b33c090518547fe0bc3c5077c2d3925978c65731ee2ed4cd1d3b43"}
Sep 10 14:41:47.041055 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.041026    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:47.078005 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.077945    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:41:47.115258 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.115201    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:47.175971 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.175936    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-maas-api-tls\") pod \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") "
Sep 10 14:41:47.176146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.176028    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5x2mc\" (UniqueName: \"kubernetes.io/projected/18778b8d-2cec-4f05-b2a2-69ff76afc41f-kube-api-access-5x2mc\") pod \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") "
Sep 10 14:41:47.176146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.176086    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18778b8d-2cec-4f05-b2a2-69ff76afc41f-openshift-service-ca\") pod \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") "
Sep 10 14:41:47.176267 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.176204    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-metrics-certs\") pod \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\" (UID: \"18778b8d-2cec-4f05-b2a2-69ff76afc41f\") "
Sep 10 14:41:47.176462 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.176438    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18778b8d-2cec-4f05-b2a2-69ff76afc41f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "18778b8d-2cec-4f05-b2a2-69ff76afc41f" (UID: "18778b8d-2cec-4f05-b2a2-69ff76afc41f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:47.176568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.176553    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/18778b8d-2cec-4f05-b2a2-69ff76afc41f-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:47.178293 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.178268    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "18778b8d-2cec-4f05-b2a2-69ff76afc41f" (UID: "18778b8d-2cec-4f05-b2a2-69ff76afc41f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:47.178372 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.178312    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18778b8d-2cec-4f05-b2a2-69ff76afc41f-kube-api-access-5x2mc" (OuterVolumeSpecName: "kube-api-access-5x2mc") pod "18778b8d-2cec-4f05-b2a2-69ff76afc41f" (UID: "18778b8d-2cec-4f05-b2a2-69ff76afc41f"). InnerVolumeSpecName "kube-api-access-5x2mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:47.178430 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.178380    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "18778b8d-2cec-4f05-b2a2-69ff76afc41f" (UID: "18778b8d-2cec-4f05-b2a2-69ff76afc41f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:47.277295 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.277255    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:47.277295 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.277287    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/18778b8d-2cec-4f05-b2a2-69ff76afc41f-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:47.277295 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.277300    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5x2mc\" (UniqueName: \"kubernetes.io/projected/18778b8d-2cec-4f05-b2a2-69ff76afc41f-kube-api-access-5x2mc\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:41:47.396194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.396108    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:47.993650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.993621    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"
Sep 10 14:41:47.994053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.993623    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr" event={"ID":"18778b8d-2cec-4f05-b2a2-69ff76afc41f","Type":"ContainerDied","Data":"4b0d643856d30044e66ff00a4e2f14d82f70d328f7490d21d5c283e10c4a33e5"}
Sep 10 14:41:47.994053 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:47.993755    2576 scope.go:117] "RemoveContainer" containerID="dc719231f6b33c090518547fe0bc3c5077c2d3925978c65731ee2ed4cd1d3b43"
Sep 10 14:41:48.014407 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:48.014380    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"]
Sep 10 14:41:48.018358 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:48.018336    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-90366e-6d4c49dd69-25gqr"]
Sep 10 14:41:48.427674 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:48.427642    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18778b8d-2cec-4f05-b2a2-69ff76afc41f" path="/var/lib/kubelet/pods/18778b8d-2cec-4f05-b2a2-69ff76afc41f/volumes"
Sep 10 14:41:52.376479 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:52.376392    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"]
Sep 10 14:41:52.422059 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:52.422017    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:57.216872 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:57.216837    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:41:58.458571 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:58.458539    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:41:58.478417 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:41:58.478396    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:42:07.262506 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.262454    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:07.396990 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.396924    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c" podUID="2d53dda8-9f3c-4dd3-b228-426520874059" containerName="main" containerID="cri-o://9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59" gracePeriod=45
Sep 10 14:42:07.651243 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.651206    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:42:07.656633 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656615    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-model-cache\") pod \"2d53dda8-9f3c-4dd3-b228-426520874059\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") "
Sep 10 14:42:07.656706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656648    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-home\") pod \"2d53dda8-9f3c-4dd3-b228-426520874059\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") "
Sep 10 14:42:07.656706 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656673    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d53dda8-9f3c-4dd3-b228-426520874059-tls-certs\") pod \"2d53dda8-9f3c-4dd3-b228-426520874059\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") "
Sep 10 14:42:07.656776 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656706    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-dshm\") pod \"2d53dda8-9f3c-4dd3-b228-426520874059\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") "
Sep 10 14:42:07.656776 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656755    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hglvq\" (UniqueName: \"kubernetes.io/projected/2d53dda8-9f3c-4dd3-b228-426520874059-kube-api-access-hglvq\") pod \"2d53dda8-9f3c-4dd3-b228-426520874059\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") "
Sep 10 14:42:07.656852 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656827    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-tmp-dir\") pod \"2d53dda8-9f3c-4dd3-b228-426520874059\" (UID: \"2d53dda8-9f3c-4dd3-b228-426520874059\") "
Sep 10 14:42:07.656905 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656861    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-model-cache" (OuterVolumeSpecName: "model-cache") pod "2d53dda8-9f3c-4dd3-b228-426520874059" (UID: "2d53dda8-9f3c-4dd3-b228-426520874059"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:42:07.656962 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.656934    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-home" (OuterVolumeSpecName: "home") pod "2d53dda8-9f3c-4dd3-b228-426520874059" (UID: "2d53dda8-9f3c-4dd3-b228-426520874059"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:42:07.657138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.657120    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:07.657138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.657118    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2d53dda8-9f3c-4dd3-b228-426520874059" (UID: "2d53dda8-9f3c-4dd3-b228-426520874059"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:42:07.657138 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.657138    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:07.658991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.658963    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-dshm" (OuterVolumeSpecName: "dshm") pod "2d53dda8-9f3c-4dd3-b228-426520874059" (UID: "2d53dda8-9f3c-4dd3-b228-426520874059"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:42:07.659093 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.659029    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d53dda8-9f3c-4dd3-b228-426520874059-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2d53dda8-9f3c-4dd3-b228-426520874059" (UID: "2d53dda8-9f3c-4dd3-b228-426520874059"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:07.659161 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.659096    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d53dda8-9f3c-4dd3-b228-426520874059-kube-api-access-hglvq" (OuterVolumeSpecName: "kube-api-access-hglvq") pod "2d53dda8-9f3c-4dd3-b228-426520874059" (UID: "2d53dda8-9f3c-4dd3-b228-426520874059"). InnerVolumeSpecName "kube-api-access-hglvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:07.757612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.757575    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:07.757612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.757605    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2d53dda8-9f3c-4dd3-b228-426520874059-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:07.757612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.757615    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2d53dda8-9f3c-4dd3-b228-426520874059-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:07.757850 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:07.757624    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hglvq\" (UniqueName: \"kubernetes.io/projected/2d53dda8-9f3c-4dd3-b228-426520874059-kube-api-access-hglvq\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:08.099748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.099708    2576 generic.go:356] "Generic (PLEG): container finished" podID="2d53dda8-9f3c-4dd3-b228-426520874059" containerID="9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59" exitCode=0
Sep 10 14:42:08.099748 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.099752    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"
Sep 10 14:42:08.099958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.099762    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c" event={"ID":"2d53dda8-9f3c-4dd3-b228-426520874059","Type":"ContainerDied","Data":"9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59"}
Sep 10 14:42:08.099958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.099802    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c" event={"ID":"2d53dda8-9f3c-4dd3-b228-426520874059","Type":"ContainerDied","Data":"dd4af557cd23e1daa9c16fe7aa0d48fe315a9a7d8510c5562a47626ab942381e"}
Sep 10 14:42:08.099958 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.099824    2576 scope.go:117] "RemoveContainer" containerID="9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59"
Sep 10 14:42:08.108904 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.108885    2576 scope.go:117] "RemoveContainer" containerID="9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59"
Sep 10 14:42:08.109175 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:42:08.109156    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59\": container with ID starting with 9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59 not found: ID does not exist" containerID="9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59"
Sep 10 14:42:08.109260 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.109184    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59"} err="failed to get container status \"9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59\": rpc error: code = NotFound desc = could not find container \"9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59\": container with ID starting with 9cf58504d40b17605449181b0c8aff5061dbc9653bc3cdf4bdc32fd8969b3f59 not found: ID does not exist"
Sep 10 14:42:08.126202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.126176    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"]
Sep 10 14:42:08.129343 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.129320    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-90366e/ipp-route-90366e-kserve-6fdfdb8d6c-9hh6c"]
Sep 10 14:42:08.427420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:08.427388    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d53dda8-9f3c-4dd3-b228-426520874059" path="/var/lib/kubelet/pods/2d53dda8-9f3c-4dd3-b228-426520874059/volumes"
Sep 10 14:42:26.191163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:26.191099    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:26.246907 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:26.246876    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:29.182942 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:29.182911    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:29.479169 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:29.479072    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:35.048121 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:35.048085    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:35.425401 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:35.425371    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:40.660417 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:40.660382    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:40.730285 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:40.730252    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:40.976362 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:40.976327    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:41.385746 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:41.385652    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:51.658024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.657992    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"]
Sep 10 14:42:51.658639 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.658221    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" podUID="a73722b1-d370-4ec8-85e5-c7053443427e" containerName="maas-api" containerID="cri-o://b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a" gracePeriod=30
Sep 10 14:42:51.686286 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.686226    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:51.757862 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.757828    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:51.858259 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.858199    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:51.914912 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.914850    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:42:51.920579 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.920553    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:51.962719 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.962693    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4zwx\" (UniqueName: \"kubernetes.io/projected/a73722b1-d370-4ec8-85e5-c7053443427e-kube-api-access-t4zwx\") pod \"a73722b1-d370-4ec8-85e5-c7053443427e\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") "
Sep 10 14:42:51.962887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.962749    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls\") pod \"a73722b1-d370-4ec8-85e5-c7053443427e\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") "
Sep 10 14:42:51.962887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.962839    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-metrics-certs\") pod \"a73722b1-d370-4ec8-85e5-c7053443427e\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") "
Sep 10 14:42:51.963021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.962963    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73722b1-d370-4ec8-85e5-c7053443427e-openshift-service-ca\") pod \"a73722b1-d370-4ec8-85e5-c7053443427e\" (UID: \"a73722b1-d370-4ec8-85e5-c7053443427e\") "
Sep 10 14:42:51.963393 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.963362    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a73722b1-d370-4ec8-85e5-c7053443427e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a73722b1-d370-4ec8-85e5-c7053443427e" (UID: "a73722b1-d370-4ec8-85e5-c7053443427e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:42:51.965073 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.965044    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a73722b1-d370-4ec8-85e5-c7053443427e-kube-api-access-t4zwx" (OuterVolumeSpecName: "kube-api-access-t4zwx") pod "a73722b1-d370-4ec8-85e5-c7053443427e" (UID: "a73722b1-d370-4ec8-85e5-c7053443427e"). InnerVolumeSpecName "kube-api-access-t4zwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:51.965558 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.965533    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a73722b1-d370-4ec8-85e5-c7053443427e" (UID: "a73722b1-d370-4ec8-85e5-c7053443427e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:51.965668 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:51.965560    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a73722b1-d370-4ec8-85e5-c7053443427e" (UID: "a73722b1-d370-4ec8-85e5-c7053443427e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:52.064271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.064214    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4zwx\" (UniqueName: \"kubernetes.io/projected/a73722b1-d370-4ec8-85e5-c7053443427e-kube-api-access-t4zwx\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:52.064271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.064266    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:52.064271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.064275    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a73722b1-d370-4ec8-85e5-c7053443427e-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:52.064271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.064283    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a73722b1-d370-4ec8-85e5-c7053443427e-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:42:52.281545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.281513    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:52.310659 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.310633    2576 generic.go:356] "Generic (PLEG): container finished" podID="a73722b1-d370-4ec8-85e5-c7053443427e" containerID="b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a" exitCode=0
Sep 10 14:42:52.310794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.310679    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"
Sep 10 14:42:52.310794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.310697    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" event={"ID":"a73722b1-d370-4ec8-85e5-c7053443427e","Type":"ContainerDied","Data":"b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a"}
Sep 10 14:42:52.310794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.310752    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44" event={"ID":"a73722b1-d370-4ec8-85e5-c7053443427e","Type":"ContainerDied","Data":"8aef8fda202f3d35a508a8e58c530fa281c4032d37ea3ef4da7a075245b89c13"}
Sep 10 14:42:52.310794 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.310781    2576 scope.go:117] "RemoveContainer" containerID="b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a"
Sep 10 14:42:52.320394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.320375    2576 scope.go:117] "RemoveContainer" containerID="b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a"
Sep 10 14:42:52.320678 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:42:52.320659    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a\": container with ID starting with b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a not found: ID does not exist" containerID="b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a"
Sep 10 14:42:52.320726 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.320685    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a"} err="failed to get container status \"b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a\": rpc error: code = NotFound desc = could not find container \"b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a\": container with ID starting with b01492a4e241793021e256255bbf928cb2fb66abd0c633dfcb57ae5492c91e9a not found: ID does not exist"
Sep 10 14:42:52.333649 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.333623    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"]
Sep 10 14:42:52.339620 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.339598    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-6b9e31-785c5678d9-zwv44"]
Sep 10 14:42:52.427545 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:52.427511    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a73722b1-d370-4ec8-85e5-c7053443427e" path="/var/lib/kubelet/pods/a73722b1-d370-4ec8-85e5-c7053443427e/volumes"
Sep 10 14:42:55.218691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:55.218658    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:42:58.101098 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:58.101057    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:58.161500 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:58.161469    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:58.347013 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:58.346983    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:58.800797 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:42:58.800767    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:09.126922 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:09.126891    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:09.226683 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:09.226630    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:09.292595 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:09.292563    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:09.345193 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:09.345159    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:09.666511 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:09.666475    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:12.212331 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:12.212291    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:43:15.382270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:15.382210    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:15.446881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:15.446845    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:15.626851 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:15.626816    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"]
Sep 10 14:43:15.627491 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:15.627455    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" containerID="cri-o://9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30" gracePeriod=30
Sep 10 14:43:15.647505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:15.647443    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:16.116485 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:16.116453    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:20.428356 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:20.428270    2576 patch_prober.go:28] interesting pod/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 14:43:20.428356 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:20.428335    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:43:20.892209 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:20.892183    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:43:21.041420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041326    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-socket\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041371    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-data\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041417    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-certs\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041445    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-istio-token\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041473    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm794\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-kube-api-access-xm794\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041522    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8efc1563-83e0-4080-abd9-f219a5231e01-istiod-ca-cert\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041549    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-envoy\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041581    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-credential-socket\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041611    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8efc1563-83e0-4080-abd9-f219a5231e01-istio-podinfo\") pod \"8efc1563-83e0-4080-abd9-f219a5231e01\" (UID: \"8efc1563-83e0-4080-abd9-f219a5231e01\") "
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041653    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041673    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:21.041722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041686    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-data" (OuterVolumeSpecName: "istio-data") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:21.042202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041924    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.042202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041952    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.042202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041965    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.042202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041924    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:21.042202 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.041952    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8efc1563-83e0-4080-abd9-f219a5231e01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:21.043981 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.043959    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8efc1563-83e0-4080-abd9-f219a5231e01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:43:21.044087 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.043992    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-istio-token" (OuterVolumeSpecName: "istio-token") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:21.044425 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.044399    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:21.044425 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.044410    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-kube-api-access-xm794" (OuterVolumeSpecName: "kube-api-access-xm794") pod "8efc1563-83e0-4080-abd9-f219a5231e01" (UID: "8efc1563-83e0-4080-abd9-f219a5231e01"). InnerVolumeSpecName "kube-api-access-xm794". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:21.142831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.142797    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm794\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-kube-api-access-xm794\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.142831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.142826    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8efc1563-83e0-4080-abd9-f219a5231e01-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.142831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.142835    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.143064 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.142843    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8efc1563-83e0-4080-abd9-f219a5231e01-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.143064 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.142851    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8efc1563-83e0-4080-abd9-f219a5231e01-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.143064 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.142859    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8efc1563-83e0-4080-abd9-f219a5231e01-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:21.451039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.451008    2576 generic.go:356] "Generic (PLEG): container finished" podID="8efc1563-83e0-4080-abd9-f219a5231e01" containerID="9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30" exitCode=0
Sep 10 14:43:21.451490 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.451064    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"
Sep 10 14:43:21.451490 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.451077    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" event={"ID":"8efc1563-83e0-4080-abd9-f219a5231e01","Type":"ContainerDied","Data":"9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30"}
Sep 10 14:43:21.451490 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.451125    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8" event={"ID":"8efc1563-83e0-4080-abd9-f219a5231e01","Type":"ContainerDied","Data":"547eca6a3217037cb41b9ae5fc2011999f40447002ba5564ed67eea175181000"}
Sep 10 14:43:21.451490 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.451147    2576 scope.go:117] "RemoveContainer" containerID="9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30"
Sep 10 14:43:21.460475 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.460456    2576 scope.go:117] "RemoveContainer" containerID="e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"
Sep 10 14:43:21.468157 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.468140    2576 scope.go:117] "RemoveContainer" containerID="9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30"
Sep 10 14:43:21.468422 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:43:21.468403    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30\": container with ID starting with 9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30 not found: ID does not exist" containerID="9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30"
Sep 10 14:43:21.468469 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.468431    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30"} err="failed to get container status \"9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30\": rpc error: code = NotFound desc = could not find container \"9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30\": container with ID starting with 9d37c27ba91a94457167ce9ef27caff7cb414705ce4287f44ecd2256c0740b30 not found: ID does not exist"
Sep 10 14:43:21.468469 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.468445    2576 scope.go:117] "RemoveContainer" containerID="e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"
Sep 10 14:43:21.468640 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:43:21.468625    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698\": container with ID starting with e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698 not found: ID does not exist" containerID="e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"
Sep 10 14:43:21.468681 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.468644    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698"} err="failed to get container status \"e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698\": rpc error: code = NotFound desc = could not find container \"e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698\": container with ID starting with e90987df418b47641d4cc0bee810c21b13c60b6258013e3b66c6e102f57fb698 not found: ID does not exist"
Sep 10 14:43:21.476194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.474572    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"]
Sep 10 14:43:21.480650 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:21.480626    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-586826-openshift-default-7d5dc7f994-hntb8"]
Sep 10 14:43:22.427010 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:22.426970    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" path="/var/lib/kubelet/pods/8efc1563-83e0-4080-abd9-f219a5231e01/volumes"
Sep 10 14:43:26.392289 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:26.392205    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:26.456398 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:26.456360    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:26.516066 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:26.516027    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:26.593672 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:26.593631    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:27.032621 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:27.032589    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:29.021729 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:29.021680    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:43:32.768299 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:32.768263    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:32.907012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:32.906976    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:32.991464 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:32.991433    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"]
Sep 10 14:43:32.991662 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:32.991640    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" containerID="cri-o://9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889" gracePeriod=30
Sep 10 14:43:33.013803 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:33.013768    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:35.478683 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:35.478652    2576 patch_prober.go:28] interesting pod/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 10 14:43:35.479048 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:35.478718    2576 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 10 14:43:38.009911 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.009876    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:43:38.256707 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.256649    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:43:38.292384 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292350    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn6jb\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-kube-api-access-jn6jb\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292410    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50f2c681-ce81-4278-926a-d08b2f5a2929-istiod-ca-cert\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292439    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-credential-socket\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292480    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-data\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292494    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-certs\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292708 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292539    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-socket\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292708 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292580    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-envoy\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292708 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292606    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-token\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292708 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292643    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-podinfo\") pod \"50f2c681-ce81-4278-926a-d08b2f5a2929\" (UID: \"50f2c681-ce81-4278-926a-d08b2f5a2929\") "
Sep 10 14:43:38.292914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292725    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:38.292914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292771    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-data" (OuterVolumeSpecName: "istio-data") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:38.292914 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292816    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50f2c681-ce81-4278-926a-d08b2f5a2929-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:38.293080 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.292963    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:38.293080 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.293054    2576 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-data\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.293080 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.293072    2576 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.293203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.293084    2576 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/50f2c681-ce81-4278-926a-d08b2f5a2929-istiod-ca-cert\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.293203 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.293098    2576 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-credential-socket\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.293597 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.293566    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:38.294871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.294846    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-kube-api-access-jn6jb" (OuterVolumeSpecName: "kube-api-access-jn6jb") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "kube-api-access-jn6jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:38.295531 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.295501    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 10 14:43:38.295627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.295593    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:43:38.295670 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.295622    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-token" (OuterVolumeSpecName: "istio-token") pod "50f2c681-ce81-4278-926a-d08b2f5a2929" (UID: "50f2c681-ce81-4278-926a-d08b2f5a2929"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:38.393881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.393849    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn6jb\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-kube-api-access-jn6jb\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.393881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.393876    2576 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-workload-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.393881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.393887    2576 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-envoy\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.394264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.393895    2576 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-token\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.394264 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.393903    2576 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/50f2c681-ce81-4278-926a-d08b2f5a2929-istio-podinfo\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:43:38.533305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.533193    2576 generic.go:356] "Generic (PLEG): container finished" podID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerID="9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889" exitCode=0
Sep 10 14:43:38.533305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.533259    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"
Sep 10 14:43:38.533305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.533276    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" event={"ID":"50f2c681-ce81-4278-926a-d08b2f5a2929","Type":"ContainerDied","Data":"9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889"}
Sep 10 14:43:38.533548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.533332    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7" event={"ID":"50f2c681-ce81-4278-926a-d08b2f5a2929","Type":"ContainerDied","Data":"419ca124018818a64f90514f7a4cf8e1b7495360e4e3442ae24d318c4355f6e5"}
Sep 10 14:43:38.533548 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.533354    2576 scope.go:117] "RemoveContainer" containerID="9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889"
Sep 10 14:43:38.542567 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.542551    2576 scope.go:117] "RemoveContainer" containerID="da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"
Sep 10 14:43:38.551163 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.551133    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"]
Sep 10 14:43:38.552271 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.552254    2576 scope.go:117] "RemoveContainer" containerID="9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889"
Sep 10 14:43:38.552756 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:43:38.552730    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889\": container with ID starting with 9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889 not found: ID does not exist" containerID="9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889"
Sep 10 14:43:38.552891 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.552869    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889"} err="failed to get container status \"9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889\": rpc error: code = NotFound desc = could not find container \"9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889\": container with ID starting with 9053a5197fe5e04266fa5091c23cfdd929994bd2c8f7d0c2289f982f5bf08889 not found: ID does not exist"
Sep 10 14:43:38.552985 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.552973    2576 scope.go:117] "RemoveContainer" containerID="da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"
Sep 10 14:43:38.553395 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:43:38.553360    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9\": container with ID starting with da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9 not found: ID does not exist" containerID="da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"
Sep 10 14:43:38.553477 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.553395    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9"} err="failed to get container status \"da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9\": rpc error: code = NotFound desc = could not find container \"da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9\": container with ID starting with da1c4e9cf38b18183e5a2d48bd8e1796b678540baf950fa0e629a7466d2150a9 not found: ID does not exist"
Sep 10 14:43:38.556146 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:38.556126    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-9dcc39-openshift-default-6d69d9fbdd-5jnf7"]
Sep 10 14:43:40.427156 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:40.427112    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" path="/var/lib/kubelet/pods/50f2c681-ce81-4278-926a-d08b2f5a2929/volumes"
Sep 10 14:43:51.321411 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:51.321368    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:52.902501 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:43:52.902467    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:57.166881 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:44:57.166839    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:57.907265 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:44:57.907218    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:44:59.039130 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:44:59.039075    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:00.618039 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:45:00.618004    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:25.612420 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:45:25.612388    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:45:32.703702 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:45:32.703660    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:33.769097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:45:33.769057    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:45:41.618926 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:45:41.618887    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:45:54.805861 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:45:54.805824    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:46:01.916498 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:01.916456    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:46:04.211313 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:04.211279    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"]
Sep 10 14:46:04.211726 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:04.211506    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g" podUID="75a5368a-83aa-413c-a287-d987cbd312f4" containerName="manager" containerID="cri-o://220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848" gracePeriod=10
Sep 10 14:46:05.367689 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.367662    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:46:05.478313 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.478275    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75a5368a-83aa-413c-a287-d987cbd312f4-extensions-socket-volume\") pod \"75a5368a-83aa-413c-a287-d987cbd312f4\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") "
Sep 10 14:46:05.478313 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.478320    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h66q5\" (UniqueName: \"kubernetes.io/projected/75a5368a-83aa-413c-a287-d987cbd312f4-kube-api-access-h66q5\") pod \"75a5368a-83aa-413c-a287-d987cbd312f4\" (UID: \"75a5368a-83aa-413c-a287-d987cbd312f4\") "
Sep 10 14:46:05.478679 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.478655    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a5368a-83aa-413c-a287-d987cbd312f4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "75a5368a-83aa-413c-a287-d987cbd312f4" (UID: "75a5368a-83aa-413c-a287-d987cbd312f4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:46:05.480620 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.480589    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a5368a-83aa-413c-a287-d987cbd312f4-kube-api-access-h66q5" (OuterVolumeSpecName: "kube-api-access-h66q5") pod "75a5368a-83aa-413c-a287-d987cbd312f4" (UID: "75a5368a-83aa-413c-a287-d987cbd312f4"). InnerVolumeSpecName "kube-api-access-h66q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:05.579871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.579835    2576 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/75a5368a-83aa-413c-a287-d987cbd312f4-extensions-socket-volume\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:46:05.579871 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:05.579863    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h66q5\" (UniqueName: \"kubernetes.io/projected/75a5368a-83aa-413c-a287-d987cbd312f4-kube-api-access-h66q5\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:46:06.216684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.216647    2576 generic.go:356] "Generic (PLEG): container finished" podID="75a5368a-83aa-413c-a287-d987cbd312f4" containerID="220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848" exitCode=0
Sep 10 14:46:06.216887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.216712    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"
Sep 10 14:46:06.216887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.216734    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g" event={"ID":"75a5368a-83aa-413c-a287-d987cbd312f4","Type":"ContainerDied","Data":"220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848"}
Sep 10 14:46:06.216887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.216820    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g" event={"ID":"75a5368a-83aa-413c-a287-d987cbd312f4","Type":"ContainerDied","Data":"ffd03729b7455b504d101f8c66166e24fd39ed663d62dd950259b61c291e36c3"}
Sep 10 14:46:06.216887 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.216853    2576 scope.go:117] "RemoveContainer" containerID="220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848"
Sep 10 14:46:06.226136 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.226118    2576 scope.go:117] "RemoveContainer" containerID="220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848"
Sep 10 14:46:06.226448 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:46:06.226428    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848\": container with ID starting with 220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848 not found: ID does not exist" containerID="220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848"
Sep 10 14:46:06.226516 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.226456    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848"} err="failed to get container status \"220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848\": rpc error: code = NotFound desc = could not find container \"220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848\": container with ID starting with 220bd70e839909eda27a3e198e162017ee2c7ab2783307e1f3ab07982440a848 not found: ID does not exist"
Sep 10 14:46:06.246217 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.246195    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"]
Sep 10 14:46:06.253438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.253418    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-q792g"]
Sep 10 14:46:06.426615 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:06.426580    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a5368a-83aa-413c-a287-d987cbd312f4" path="/var/lib/kubelet/pods/75a5368a-83aa-413c-a287-d987cbd312f4/volumes"
Sep 10 14:46:58.484610 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:58.484585    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:46:58.504187 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:46:58.504162    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:47:04.866889 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.866848    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"]
Sep 10 14:47:04.867413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867355    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d53dda8-9f3c-4dd3-b228-426520874059" containerName="main"
Sep 10 14:47:04.867413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867373    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d53dda8-9f3c-4dd3-b228-426520874059" containerName="main"
Sep 10 14:47:04.867413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867384    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy"
Sep 10 14:47:04.867413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867392    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy"
Sep 10 14:47:04.867413 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867412    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a5368a-83aa-413c-a287-d987cbd312f4" containerName="manager"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867421    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a5368a-83aa-413c-a287-d987cbd312f4" containerName="manager"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867432    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867439    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867454    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867462    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867475    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867483    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867491    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18778b8d-2cec-4f05-b2a2-69ff76afc41f" containerName="maas-api"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867499    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="18778b8d-2cec-4f05-b2a2-69ff76afc41f" containerName="maas-api"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867517    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867527    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867535    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a73722b1-d370-4ec8-85e5-c7053443427e" containerName="maas-api"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867543    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="a73722b1-d370-4ec8-85e5-c7053443427e" containerName="maas-api"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867624    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a5368a-83aa-413c-a287-d987cbd312f4" containerName="manager"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867637    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867648    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867657    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d014f0-f76d-4e7d-8fe6-5740126c9deb" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867666    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="a73722b1-d370-4ec8-85e5-c7053443427e" containerName="maas-api"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867676    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867684    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="50f2c681-ce81-4278-926a-d08b2f5a2929" containerName="istio-proxy"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867691    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="18778b8d-2cec-4f05-b2a2-69ff76afc41f" containerName="maas-api"
Sep 10 14:47:04.867693 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.867700    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d53dda8-9f3c-4dd3-b228-426520874059" containerName="main"
Sep 10 14:47:04.870740 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.870716    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:04.873485 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.873469    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-j42sg\""
Sep 10 14:47:04.882713 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.882691    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"]
Sep 10 14:47:04.956542 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.956509    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkhfk\" (UniqueName: \"kubernetes.io/projected/7effa53a-5c45-401d-82fb-e3be4b577cb2-kube-api-access-hkhfk\") pod \"kuadrant-operator-controller-manager-5769f99d9f-j2wb6\" (UID: \"7effa53a-5c45-401d-82fb-e3be4b577cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:04.956696 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:04.956570    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7effa53a-5c45-401d-82fb-e3be4b577cb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-j2wb6\" (UID: \"7effa53a-5c45-401d-82fb-e3be4b577cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:05.057604 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.057569    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7effa53a-5c45-401d-82fb-e3be4b577cb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-j2wb6\" (UID: \"7effa53a-5c45-401d-82fb-e3be4b577cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:05.057735 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.057651    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkhfk\" (UniqueName: \"kubernetes.io/projected/7effa53a-5c45-401d-82fb-e3be4b577cb2-kube-api-access-hkhfk\") pod \"kuadrant-operator-controller-manager-5769f99d9f-j2wb6\" (UID: \"7effa53a-5c45-401d-82fb-e3be4b577cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:05.057934 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.057917    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7effa53a-5c45-401d-82fb-e3be4b577cb2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-j2wb6\" (UID: \"7effa53a-5c45-401d-82fb-e3be4b577cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:05.066368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.066342    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkhfk\" (UniqueName: \"kubernetes.io/projected/7effa53a-5c45-401d-82fb-e3be4b577cb2-kube-api-access-hkhfk\") pod \"kuadrant-operator-controller-manager-5769f99d9f-j2wb6\" (UID: \"7effa53a-5c45-401d-82fb-e3be4b577cb2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:05.181270 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.181222    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:05.475209 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.475111    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"]
Sep 10 14:47:05.478724 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:47:05.478697    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7effa53a_5c45_401d_82fb_e3be4b577cb2.slice/crio-3ef06960b621d749160666ca2867759fd3e7ab91ce425a3791fe24105f21059f WatchSource:0}: Error finding container 3ef06960b621d749160666ca2867759fd3e7ab91ce425a3791fe24105f21059f: Status 404 returned error can't find the container with id 3ef06960b621d749160666ca2867759fd3e7ab91ce425a3791fe24105f21059f
Sep 10 14:47:05.491753 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:05.491714    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6" event={"ID":"7effa53a-5c45-401d-82fb-e3be4b577cb2","Type":"ContainerStarted","Data":"3ef06960b621d749160666ca2867759fd3e7ab91ce425a3791fe24105f21059f"}
Sep 10 14:47:06.498070 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:06.498035    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6" event={"ID":"7effa53a-5c45-401d-82fb-e3be4b577cb2","Type":"ContainerStarted","Data":"ca5e28c1f70d49ee9c60f0c2d2d7f8e42f44e04d613fad2bcb915bd0651bb1fe"}
Sep 10 14:47:06.498540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:06.498161    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:06.524017 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:06.523969    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6" podStartSLOduration=2.523953732 podStartE2EDuration="2.523953732s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:06.521397738 +0000 UTC m=+2108.609855310" watchObservedRunningTime="2026-09-10 14:47:06.523953732 +0000 UTC m=+2108.612411304"
Sep 10 14:47:17.504989 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:17.504950    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-j2wb6"
Sep 10 14:47:24.711483 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:24.711445    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:47:30.406609 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:30.406571    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:47:37.713936 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:37.713901    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:47:39.607568 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.607534    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"]
Sep 10 14:47:39.607939 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.607870    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy"
Sep 10 14:47:39.607939 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.607884    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy"
Sep 10 14:47:39.608031 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.607976    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="8efc1563-83e0-4080-abd9-f219a5231e01" containerName="istio-proxy"
Sep 10 14:47:39.611016 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.610990    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.614055 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.614031    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-f5652026-kserve-self-signed-certs\""
Sep 10 14:47:39.620090 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.620067    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"]
Sep 10 14:47:39.667102 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.667067    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:39.777831 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.777798    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdf4919-4212-481e-bd20-a000d9df55cb-tls-certs\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.778043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.777839    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-model-cache\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.778043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.777862    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8lbds\" (UniqueName: \"kubernetes.io/projected/2bdf4919-4212-481e-bd20-a000d9df55cb-kube-api-access-8lbds\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.778043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.777931    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-dshm\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.778043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.777954    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-tmp-dir\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.778043 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.777981    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-home\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879008    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdf4919-4212-481e-bd20-a000d9df55cb-tls-certs\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879055    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-model-cache\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879188    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8lbds\" (UniqueName: \"kubernetes.io/projected/2bdf4919-4212-481e-bd20-a000d9df55cb-kube-api-access-8lbds\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879255    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-dshm\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879281    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-tmp-dir\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879317    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-home\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879608 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879524    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-model-cache\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879682 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879656    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-tmp-dir\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.879736 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.879696    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-home\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.881692 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.881666    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-dshm\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.881814 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.881799    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdf4919-4212-481e-bd20-a000d9df55cb-tls-certs\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.886760 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.886740    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8lbds\" (UniqueName: \"kubernetes.io/projected/2bdf4919-4212-481e-bd20-a000d9df55cb-kube-api-access-8lbds\") pod \"e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") " pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:39.920839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:39.920804    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:40.192260 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:40.192220    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"]
Sep 10 14:47:40.195509 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:47:40.195468    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2bdf4919_4212_481e_bd20_a000d9df55cb.slice/crio-f6a3e9cb5c2c0d8d744e44030d506c6f56b63f0dc756d86d9db38b142d5f03cd WatchSource:0}: Error finding container f6a3e9cb5c2c0d8d744e44030d506c6f56b63f0dc756d86d9db38b142d5f03cd: Status 404 returned error can't find the container with id f6a3e9cb5c2c0d8d744e44030d506c6f56b63f0dc756d86d9db38b142d5f03cd
Sep 10 14:47:40.660731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:40.660645    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8" event={"ID":"2bdf4919-4212-481e-bd20-a000d9df55cb","Type":"ContainerStarted","Data":"ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205"}
Sep 10 14:47:40.660731 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:40.660695    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8" event={"ID":"2bdf4919-4212-481e-bd20-a000d9df55cb","Type":"ContainerStarted","Data":"f6a3e9cb5c2c0d8d744e44030d506c6f56b63f0dc756d86d9db38b142d5f03cd"}
Sep 10 14:47:40.677105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:40.677055    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8" podStartSLOduration=1.6770387979999999 podStartE2EDuration="1.677038798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:40.676072125 +0000 UTC m=+2142.764529696" watchObservedRunningTime="2026-09-10 14:47:40.677038798 +0000 UTC m=+2142.765496369"
Sep 10 14:47:45.907860 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:45.907826    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:47:47.328304 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.328274    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"]
Sep 10 14:47:47.331820 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.331801    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.334438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.334412    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-f5652026-kserve-self-signed-certs\""
Sep 10 14:47:47.342769 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.342744    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-model-cache\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.342886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.342810    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-tmp-dir\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.342886 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.342843    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlcq6\" (UniqueName: \"kubernetes.io/projected/ff9ebd13-281b-41da-a405-dff427510fc3-kube-api-access-tlcq6\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.343000 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.342886    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9ebd13-281b-41da-a405-dff427510fc3-tls-certs\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.343000 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.342935    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-dshm\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.343094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.343002    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-home\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.343505 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.343482    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"]
Sep 10 14:47:47.412566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.412537    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:47.444248 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444197    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-model-cache\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444543 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444521    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-tmp-dir\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444567    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-model-cache\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444617 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444572    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlcq6\" (UniqueName: \"kubernetes.io/projected/ff9ebd13-281b-41da-a405-dff427510fc3-kube-api-access-tlcq6\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444752 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444633    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9ebd13-281b-41da-a405-dff427510fc3-tls-certs\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444752 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444667    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-dshm\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444752 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444690    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-home\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.444904 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444811    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-tmp-dir\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.445019 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.444995    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-home\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.447000 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.446980    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-dshm\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.447149 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.447132    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9ebd13-281b-41da-a405-dff427510fc3-tls-certs\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.457120 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.457098    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlcq6\" (UniqueName: \"kubernetes.io/projected/ff9ebd13-281b-41da-a405-dff427510fc3-kube-api-access-tlcq6\") pod \"e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") " pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.641438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.641347    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:47.914209 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:47.914113    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"]
Sep 10 14:47:47.918912 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:47:47.918884    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff9ebd13_281b_41da_a405_dff427510fc3.slice/crio-381e43b4946e35aa4ef30afb6f2f678820790f5610b8a254315c02d47996c169 WatchSource:0}: Error finding container 381e43b4946e35aa4ef30afb6f2f678820790f5610b8a254315c02d47996c169: Status 404 returned error can't find the container with id 381e43b4946e35aa4ef30afb6f2f678820790f5610b8a254315c02d47996c169
Sep 10 14:47:48.702220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:48.702181    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt" event={"ID":"ff9ebd13-281b-41da-a405-dff427510fc3","Type":"ContainerStarted","Data":"c887e719a98b9c264b6fdb28de3f2e1f26c0b0b1b069c47d39383a38efa04aa6"}
Sep 10 14:47:48.702220 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:48.702220    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt" event={"ID":"ff9ebd13-281b-41da-a405-dff427510fc3","Type":"ContainerStarted","Data":"381e43b4946e35aa4ef30afb6f2f678820790f5610b8a254315c02d47996c169"}
Sep 10 14:47:48.722221 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:48.722175    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt" podStartSLOduration=1.7221578690000001 podStartE2EDuration="1.722157869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:47:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:47:48.720889446 +0000 UTC m=+2150.809347016" watchObservedRunningTime="2026-09-10 14:47:48.722157869 +0000 UTC m=+2150.810615440"
Sep 10 14:47:49.693157 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:49.693112    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:49.921727 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:49.921693    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:49.922097 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:49.921743    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:49.934414 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:49.934381    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:50.722675 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:50.722649    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:47:57.011194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:57.011160    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:47:57.642412 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:57.642378    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:57.642627 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:57.642456    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:57.655535 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:57.655511    2576 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:47:57.756713 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:47:57.756683    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:48:09.130257 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:09.130204    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"]
Sep 10 14:48:09.200125 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:09.200082    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:10.702260 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:10.702214    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"]
Sep 10 14:48:10.728114 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:10.728087    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:11.308927 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:11.308895    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:48:20.421225 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:20.421175    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:48:24.150503 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.150435    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8" podUID="2bdf4919-4212-481e-bd20-a000d9df55cb" containerName="main" containerID="cri-o://ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205" gracePeriod=45
Sep 10 14:48:24.396537 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.396511    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:48:24.585869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.585836    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-home\") pod \"2bdf4919-4212-481e-bd20-a000d9df55cb\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") "
Sep 10 14:48:24.585869 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.585876    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8lbds\" (UniqueName: \"kubernetes.io/projected/2bdf4919-4212-481e-bd20-a000d9df55cb-kube-api-access-8lbds\") pod \"2bdf4919-4212-481e-bd20-a000d9df55cb\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") "
Sep 10 14:48:24.586143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.585917    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-dshm\") pod \"2bdf4919-4212-481e-bd20-a000d9df55cb\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") "
Sep 10 14:48:24.586143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.585954    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-tmp-dir\") pod \"2bdf4919-4212-481e-bd20-a000d9df55cb\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") "
Sep 10 14:48:24.586143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586004    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdf4919-4212-481e-bd20-a000d9df55cb-tls-certs\") pod \"2bdf4919-4212-481e-bd20-a000d9df55cb\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") "
Sep 10 14:48:24.586143 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586128    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-model-cache\") pod \"2bdf4919-4212-481e-bd20-a000d9df55cb\" (UID: \"2bdf4919-4212-481e-bd20-a000d9df55cb\") "
Sep 10 14:48:24.586405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586154    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-home" (OuterVolumeSpecName: "home") pod "2bdf4919-4212-481e-bd20-a000d9df55cb" (UID: "2bdf4919-4212-481e-bd20-a000d9df55cb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:24.586405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586313    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2bdf4919-4212-481e-bd20-a000d9df55cb" (UID: "2bdf4919-4212-481e-bd20-a000d9df55cb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:24.586405 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586360    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-model-cache" (OuterVolumeSpecName: "model-cache") pod "2bdf4919-4212-481e-bd20-a000d9df55cb" (UID: "2bdf4919-4212-481e-bd20-a000d9df55cb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:24.586576 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586560    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:24.586631 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586578    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:24.586631 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.586592    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:24.588370 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.588349    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bdf4919-4212-481e-bd20-a000d9df55cb-kube-api-access-8lbds" (OuterVolumeSpecName: "kube-api-access-8lbds") pod "2bdf4919-4212-481e-bd20-a000d9df55cb" (UID: "2bdf4919-4212-481e-bd20-a000d9df55cb"). InnerVolumeSpecName "kube-api-access-8lbds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:24.588466 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.588420    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-dshm" (OuterVolumeSpecName: "dshm") pod "2bdf4919-4212-481e-bd20-a000d9df55cb" (UID: "2bdf4919-4212-481e-bd20-a000d9df55cb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:24.588581 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.588567    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bdf4919-4212-481e-bd20-a000d9df55cb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2bdf4919-4212-481e-bd20-a000d9df55cb" (UID: "2bdf4919-4212-481e-bd20-a000d9df55cb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:48:24.687691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.687662    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8lbds\" (UniqueName: \"kubernetes.io/projected/2bdf4919-4212-481e-bd20-a000d9df55cb-kube-api-access-8lbds\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:24.687691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.687686    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2bdf4919-4212-481e-bd20-a000d9df55cb-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:24.687691 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.687695    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2bdf4919-4212-481e-bd20-a000d9df55cb-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:24.876690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.876606    2576 generic.go:356] "Generic (PLEG): container finished" podID="2bdf4919-4212-481e-bd20-a000d9df55cb" containerID="ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205" exitCode=0
Sep 10 14:48:24.876690 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.876667    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"
Sep 10 14:48:24.876882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.876692    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8" event={"ID":"2bdf4919-4212-481e-bd20-a000d9df55cb","Type":"ContainerDied","Data":"ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205"}
Sep 10 14:48:24.876882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.876733    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8" event={"ID":"2bdf4919-4212-481e-bd20-a000d9df55cb","Type":"ContainerDied","Data":"f6a3e9cb5c2c0d8d744e44030d506c6f56b63f0dc756d86d9db38b142d5f03cd"}
Sep 10 14:48:24.876882 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.876753    2576 scope.go:117] "RemoveContainer" containerID="ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205"
Sep 10 14:48:24.886186 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.886163    2576 scope.go:117] "RemoveContainer" containerID="ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205"
Sep 10 14:48:24.886468 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:48:24.886451    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205\": container with ID starting with ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205 not found: ID does not exist" containerID="ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205"
Sep 10 14:48:24.886522 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.886478    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205"} err="failed to get container status \"ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205\": rpc error: code = NotFound desc = could not find container \"ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205\": container with ID starting with ede5ed0a9d6cb4f448b622fc635a84cafb6c01c9a4571a76c17481634f686205 not found: ID does not exist"
Sep 10 14:48:24.897836 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.897810    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"]
Sep 10 14:48:24.901915 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:24.901896    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-f5652026-kserve-79d6fcf99-9vdj8"]
Sep 10 14:48:25.722813 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:25.722766    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt" podUID="ff9ebd13-281b-41da-a405-dff427510fc3" containerName="main" containerID="cri-o://c887e719a98b9c264b6fdb28de3f2e1f26c0b0b1b069c47d39383a38efa04aa6" gracePeriod=45
Sep 10 14:48:25.885152 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:25.885122    2576 generic.go:356] "Generic (PLEG): container finished" podID="ff9ebd13-281b-41da-a405-dff427510fc3" containerID="c887e719a98b9c264b6fdb28de3f2e1f26c0b0b1b069c47d39383a38efa04aa6" exitCode=0
Sep 10 14:48:25.885336 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:25.885199    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt" event={"ID":"ff9ebd13-281b-41da-a405-dff427510fc3","Type":"ContainerDied","Data":"c887e719a98b9c264b6fdb28de3f2e1f26c0b0b1b069c47d39383a38efa04aa6"}
Sep 10 14:48:25.974827 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:25.974767    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:48:26.000825 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.000797    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlcq6\" (UniqueName: \"kubernetes.io/projected/ff9ebd13-281b-41da-a405-dff427510fc3-kube-api-access-tlcq6\") pod \"ff9ebd13-281b-41da-a405-dff427510fc3\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") "
Sep 10 14:48:26.000989 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.000842    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-tmp-dir\") pod \"ff9ebd13-281b-41da-a405-dff427510fc3\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") "
Sep 10 14:48:26.000989 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.000868    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-home\") pod \"ff9ebd13-281b-41da-a405-dff427510fc3\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") "
Sep 10 14:48:26.000989 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.000909    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9ebd13-281b-41da-a405-dff427510fc3-tls-certs\") pod \"ff9ebd13-281b-41da-a405-dff427510fc3\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") "
Sep 10 14:48:26.001162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.000991    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-model-cache\") pod \"ff9ebd13-281b-41da-a405-dff427510fc3\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") "
Sep 10 14:48:26.001162 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.001046    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-dshm\") pod \"ff9ebd13-281b-41da-a405-dff427510fc3\" (UID: \"ff9ebd13-281b-41da-a405-dff427510fc3\") "
Sep 10 14:48:26.001293 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.001152    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-home" (OuterVolumeSpecName: "home") pod "ff9ebd13-281b-41da-a405-dff427510fc3" (UID: "ff9ebd13-281b-41da-a405-dff427510fc3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:26.001293 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.001161    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ff9ebd13-281b-41da-a405-dff427510fc3" (UID: "ff9ebd13-281b-41da-a405-dff427510fc3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:26.001397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.001322    2576 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-tmp-dir\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:26.001397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.001341    2576 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-home\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:26.001397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.001379    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-model-cache" (OuterVolumeSpecName: "model-cache") pod "ff9ebd13-281b-41da-a405-dff427510fc3" (UID: "ff9ebd13-281b-41da-a405-dff427510fc3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:26.003182 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.003149    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff9ebd13-281b-41da-a405-dff427510fc3-kube-api-access-tlcq6" (OuterVolumeSpecName: "kube-api-access-tlcq6") pod "ff9ebd13-281b-41da-a405-dff427510fc3" (UID: "ff9ebd13-281b-41da-a405-dff427510fc3"). InnerVolumeSpecName "kube-api-access-tlcq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:26.003341 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.003203    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-dshm" (OuterVolumeSpecName: "dshm") pod "ff9ebd13-281b-41da-a405-dff427510fc3" (UID: "ff9ebd13-281b-41da-a405-dff427510fc3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:48:26.003392 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.003335    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff9ebd13-281b-41da-a405-dff427510fc3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ff9ebd13-281b-41da-a405-dff427510fc3" (UID: "ff9ebd13-281b-41da-a405-dff427510fc3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:48:26.101978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.101941    2576 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ff9ebd13-281b-41da-a405-dff427510fc3-tls-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:26.101978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.101968    2576 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-model-cache\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:26.101978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.101977    2576 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ff9ebd13-281b-41da-a405-dff427510fc3-dshm\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:26.101978 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.101985    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlcq6\" (UniqueName: \"kubernetes.io/projected/ff9ebd13-281b-41da-a405-dff427510fc3-kube-api-access-tlcq6\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:26.428844 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.428805    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bdf4919-4212-481e-bd20-a000d9df55cb" path="/var/lib/kubelet/pods/2bdf4919-4212-481e-bd20-a000d9df55cb/volumes"
Sep 10 14:48:26.891373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.891280    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt" event={"ID":"ff9ebd13-281b-41da-a405-dff427510fc3","Type":"ContainerDied","Data":"381e43b4946e35aa4ef30afb6f2f678820790f5610b8a254315c02d47996c169"}
Sep 10 14:48:26.891373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.891322    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"
Sep 10 14:48:26.891373 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.891334    2576 scope.go:117] "RemoveContainer" containerID="c887e719a98b9c264b6fdb28de3f2e1f26c0b0b1b069c47d39383a38efa04aa6"
Sep 10 14:48:26.907484 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.907460    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"]
Sep 10 14:48:26.909911 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:26.909885    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-f5652026-kserve-8666cb64c5-5mkkt"]
Sep 10 14:48:28.429091 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:28.429052    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff9ebd13-281b-41da-a405-dff427510fc3" path="/var/lib/kubelet/pods/ff9ebd13-281b-41da-a405-dff427510fc3/volumes"
Sep 10 14:48:29.409446 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.409407    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:48:29.623638 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.623603    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"]
Sep 10 14:48:29.624012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.623940    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff9ebd13-281b-41da-a405-dff427510fc3" containerName="main"
Sep 10 14:48:29.624012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.623951    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff9ebd13-281b-41da-a405-dff427510fc3" containerName="main"
Sep 10 14:48:29.624012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.623971    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bdf4919-4212-481e-bd20-a000d9df55cb" containerName="main"
Sep 10 14:48:29.624012 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.623976    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bdf4919-4212-481e-bd20-a000d9df55cb" containerName="main"
Sep 10 14:48:29.624142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.624027    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bdf4919-4212-481e-bd20-a000d9df55cb" containerName="main"
Sep 10 14:48:29.624142 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.624037    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff9ebd13-281b-41da-a405-dff427510fc3" containerName="main"
Sep 10 14:48:29.628196 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.628106    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.631964 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.631941    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:48:29.632094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.631976    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-x6ssb\""
Sep 10 14:48:29.632094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.631941    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:48:29.632094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.632053    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 14:48:29.632094 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.632055    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:48:29.641319 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.641296    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"]
Sep 10 14:48:29.733932 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.733893    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-maas-api-tls\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.734129 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.733955    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ltvp\" (UniqueName: \"kubernetes.io/projected/2390c2c5-774c-4ac6-8864-35ada9f31d11-kube-api-access-9ltvp\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.734129 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.733995    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-metrics-certs\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.734129 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.734011    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2390c2c5-774c-4ac6-8864-35ada9f31d11-openshift-service-ca\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.835117 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.835071    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-metrics-certs\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.835117 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.835110    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2390c2c5-774c-4ac6-8864-35ada9f31d11-openshift-service-ca\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.835438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.835158    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-maas-api-tls\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.835438 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.835196    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ltvp\" (UniqueName: \"kubernetes.io/projected/2390c2c5-774c-4ac6-8864-35ada9f31d11-kube-api-access-9ltvp\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.835901 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.835876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2390c2c5-774c-4ac6-8864-35ada9f31d11-openshift-service-ca\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.837758 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.837728    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-maas-api-tls\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.837920 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.837902    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-metrics-certs\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.842678 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.842659    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ltvp\" (UniqueName: \"kubernetes.io/projected/2390c2c5-774c-4ac6-8864-35ada9f31d11-kube-api-access-9ltvp\") pod \"maas-api-7cd87849b-nbmhw\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") " pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:29.938666 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:29.938623    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:30.214795 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:30.214768    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"]
Sep 10 14:48:30.220917 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:30.220892    2576 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:48:30.912981 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:30.912877    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw" event={"ID":"2390c2c5-774c-4ac6-8864-35ada9f31d11","Type":"ContainerStarted","Data":"b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7"}
Sep 10 14:48:30.913445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:30.913029    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:30.913445 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:30.913049    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw" event={"ID":"2390c2c5-774c-4ac6-8864-35ada9f31d11","Type":"ContainerStarted","Data":"037eeaa4caa62ed677aeebd6ed7abede72b606b06e24026bf91cee997c3cdb57"}
Sep 10 14:48:30.930952 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:30.930901    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw" podStartSLOduration=1.5042933870000001 podStartE2EDuration="1.930884762s" totalImagesPullingTime="426.591375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:30.928904503 +0000 UTC m=+2193.017362293" watchObservedRunningTime="2026-09-10 14:48:30.930884762 +0000 UTC m=+2193.019342333"
Sep 10 14:48:34.503154 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:34.503105    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:36.923313 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:36.923285    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:37.024099 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.024067    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"]
Sep 10 14:48:37.027465 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.027448    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.039588 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.039562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"]
Sep 10 14:48:37.100681 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.100646    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3d50c8-3353-49fb-abbf-a9c832309b1e-openshift-service-ca\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.100842 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.100718    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3d50c8-3353-49fb-abbf-a9c832309b1e-metrics-certs\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.100842 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.100792    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgqx7\" (UniqueName: \"kubernetes.io/projected/ce3d50c8-3353-49fb-abbf-a9c832309b1e-kube-api-access-lgqx7\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.100937 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.100855    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3d50c8-3353-49fb-abbf-a9c832309b1e-maas-api-tls\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.202305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.202203    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3d50c8-3353-49fb-abbf-a9c832309b1e-metrics-certs\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.202305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.202297    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgqx7\" (UniqueName: \"kubernetes.io/projected/ce3d50c8-3353-49fb-abbf-a9c832309b1e-kube-api-access-lgqx7\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.202496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.202344    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3d50c8-3353-49fb-abbf-a9c832309b1e-maas-api-tls\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.202496 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.202373    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3d50c8-3353-49fb-abbf-a9c832309b1e-openshift-service-ca\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.203105 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.202998    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ce3d50c8-3353-49fb-abbf-a9c832309b1e-openshift-service-ca\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.204772 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.204752    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce3d50c8-3353-49fb-abbf-a9c832309b1e-metrics-certs\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.204878 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.204823    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ce3d50c8-3353-49fb-abbf-a9c832309b1e-maas-api-tls\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.211357 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.211334    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgqx7\" (UniqueName: \"kubernetes.io/projected/ce3d50c8-3353-49fb-abbf-a9c832309b1e-kube-api-access-lgqx7\") pod \"maas-api-c697fdc64-5pdmd\" (UID: \"ce3d50c8-3353-49fb-abbf-a9c832309b1e\") " pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.320596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.320562    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:48:37.338112 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.338083    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:37.613368 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.613343    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"]
Sep 10 14:48:37.616074 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:48:37.616045    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce3d50c8_3353_49fb_abbf_a9c832309b1e.slice/crio-2bf9250b42b350d50b071ba185b3b6a327d537a204352d3d7a9e47644a83aea7 WatchSource:0}: Error finding container 2bf9250b42b350d50b071ba185b3b6a327d537a204352d3d7a9e47644a83aea7: Status 404 returned error can't find the container with id 2bf9250b42b350d50b071ba185b3b6a327d537a204352d3d7a9e47644a83aea7
Sep 10 14:48:37.947343 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:37.947306    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd" event={"ID":"ce3d50c8-3353-49fb-abbf-a9c832309b1e","Type":"ContainerStarted","Data":"2bf9250b42b350d50b071ba185b3b6a327d537a204352d3d7a9e47644a83aea7"}
Sep 10 14:48:38.953584 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:38.953548    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd" event={"ID":"ce3d50c8-3353-49fb-abbf-a9c832309b1e","Type":"ContainerStarted","Data":"77583c3a5ae819d684e8708711abb8941b86f0e83a6a9c0b267c2d5b3dd34dc6"}
Sep 10 14:48:38.954063 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:38.953716    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:38.974342 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:38.974290    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd" podStartSLOduration=1.444743069 podStartE2EDuration="1.974273775s" totalImagesPullingTime="529.530706ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:38.972475207 +0000 UTC m=+2201.060932777" watchObservedRunningTime="2026-09-10 14:48:38.974273775 +0000 UTC m=+2201.062731346"
Sep 10 14:48:43.919663 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:43.919625    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:48:44.972877 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:44.966045    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-c697fdc64-5pdmd"
Sep 10 14:48:45.015195 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:45.015159    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"]
Sep 10 14:48:45.015455 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:45.015428    2576 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw" podUID="2390c2c5-774c-4ac6-8864-35ada9f31d11" containerName="maas-api" containerID="cri-o://b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7" gracePeriod=30
Sep 10 14:48:46.265589 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.265564    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:46.388874 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.388791    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-metrics-certs\") pod \"2390c2c5-774c-4ac6-8864-35ada9f31d11\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") "
Sep 10 14:48:46.388874 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.388841    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ltvp\" (UniqueName: \"kubernetes.io/projected/2390c2c5-774c-4ac6-8864-35ada9f31d11-kube-api-access-9ltvp\") pod \"2390c2c5-774c-4ac6-8864-35ada9f31d11\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") "
Sep 10 14:48:46.389068 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.388890    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2390c2c5-774c-4ac6-8864-35ada9f31d11-openshift-service-ca\") pod \"2390c2c5-774c-4ac6-8864-35ada9f31d11\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") "
Sep 10 14:48:46.389068 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.388984    2576 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-maas-api-tls\") pod \"2390c2c5-774c-4ac6-8864-35ada9f31d11\" (UID: \"2390c2c5-774c-4ac6-8864-35ada9f31d11\") "
Sep 10 14:48:46.389333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.389303    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2390c2c5-774c-4ac6-8864-35ada9f31d11-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2390c2c5-774c-4ac6-8864-35ada9f31d11" (UID: "2390c2c5-774c-4ac6-8864-35ada9f31d11"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:48:46.391261 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.391220    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2390c2c5-774c-4ac6-8864-35ada9f31d11-kube-api-access-9ltvp" (OuterVolumeSpecName: "kube-api-access-9ltvp") pod "2390c2c5-774c-4ac6-8864-35ada9f31d11" (UID: "2390c2c5-774c-4ac6-8864-35ada9f31d11"). InnerVolumeSpecName "kube-api-access-9ltvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:48:46.391347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.391253    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2390c2c5-774c-4ac6-8864-35ada9f31d11" (UID: "2390c2c5-774c-4ac6-8864-35ada9f31d11"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:48:46.391347 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.391270    2576 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2390c2c5-774c-4ac6-8864-35ada9f31d11" (UID: "2390c2c5-774c-4ac6-8864-35ada9f31d11"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:48:46.490580 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.490542    2576 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-maas-api-tls\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:46.490580 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.490575    2576 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2390c2c5-774c-4ac6-8864-35ada9f31d11-metrics-certs\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:46.490580 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.490584    2576 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ltvp\" (UniqueName: \"kubernetes.io/projected/2390c2c5-774c-4ac6-8864-35ada9f31d11-kube-api-access-9ltvp\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:46.490783 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.490594    2576 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2390c2c5-774c-4ac6-8864-35ada9f31d11-openshift-service-ca\") on node \"ip-10-0-128-41.ec2.internal\" DevicePath \"\""
Sep 10 14:48:46.996175 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.996139    2576 generic.go:356] "Generic (PLEG): container finished" podID="2390c2c5-774c-4ac6-8864-35ada9f31d11" containerID="b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7" exitCode=0
Sep 10 14:48:46.996360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.996190    2576 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"
Sep 10 14:48:46.996360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.996205    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw" event={"ID":"2390c2c5-774c-4ac6-8864-35ada9f31d11","Type":"ContainerDied","Data":"b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7"}
Sep 10 14:48:46.996360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.996285    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw" event={"ID":"2390c2c5-774c-4ac6-8864-35ada9f31d11","Type":"ContainerDied","Data":"037eeaa4caa62ed677aeebd6ed7abede72b606b06e24026bf91cee997c3cdb57"}
Sep 10 14:48:46.996360 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:46.996309    2576 scope.go:117] "RemoveContainer" containerID="b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7"
Sep 10 14:48:47.004991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:47.004967    2576 scope.go:117] "RemoveContainer" containerID="b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7"
Sep 10 14:48:47.005287 ip-10-0-128-41 kubenswrapper[2576]: E0910 14:48:47.005270    2576 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7\": container with ID starting with b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7 not found: ID does not exist" containerID="b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7"
Sep 10 14:48:47.005351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:47.005297    2576 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7"} err="failed to get container status \"b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7\": rpc error: code = NotFound desc = could not find container \"b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7\": container with ID starting with b128a75d54c0d71314f249016e7264101a8485f07028ae51c74382ce27d297e7 not found: ID does not exist"
Sep 10 14:48:47.012037 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:47.012010    2576 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"]
Sep 10 14:48:47.015991 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:47.015965    2576 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7cd87849b-nbmhw"]
Sep 10 14:48:48.427049 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:48.427020    2576 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2390c2c5-774c-4ac6-8864-35ada9f31d11" path="/var/lib/kubelet/pods/2390c2c5-774c-4ac6-8864-35ada9f31d11/volumes"
Sep 10 14:48:49.709092 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:49.709052    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:48:56.606103 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:48:56.606070    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-c57jv"]
Sep 10 14:49:04.506686 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:04.506642    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qw6zk_74b7d555-ab98-434a-b5b7-95688e34645a/manager/0.log"
Sep 10 14:49:04.619459 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:04.619429    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mx5dx_243380c0-5722-40b2-bf45-727be8411511/manager/0.log"
Sep 10 14:49:04.857848 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:04.857754    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dj2ng_90ece399-7522-4544-8c12-60067c925e44/server/0.log"
Sep 10 14:49:05.222928 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:05.222902    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-qgt9h_870bd46a-42b6-4a36-a639-f948e19bc0aa/copy-manifests/0.log"
Sep 10 14:49:05.229153 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:05.229131    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-qgt9h_870bd46a-42b6-4a36-a639-f948e19bc0aa/manager/0.log"
Sep 10 14:49:05.665722 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:05.665641    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-c697fdc64-5pdmd_ce3d50c8-3353-49fb-abbf-a9c832309b1e/maas-api/0.log"
Sep 10 14:49:06.943839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:06.943806    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-p5n2j_e361ffe9-56a4-4a89-8f2e-fa37702ae7ae/manager/0.log"
Sep 10 14:49:07.046732 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:07.046704    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mt5wl_c846bd41-86da-4f04-a5f9-7aedf9f95e0a/manager/0.log"
Sep 10 14:49:07.148359 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:07.148331    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-gh5th_ee23449c-2560-4f76-96fa-57b77cd7a0e0/kuadrant-console-plugin/0.log"
Sep 10 14:49:07.264935 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:07.264859    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-j2wb6_7effa53a-5c45-401d-82fb-e3be4b577cb2/manager/0.log"
Sep 10 14:49:07.372227 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:07.372199    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-c57jv_f036dde9-ef00-417a-9b32-e0cb38ca8d04/limitador/0.log"
Sep 10 14:49:07.479461 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:07.479429    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-pmdgx_1f8014f5-448d-4fcd-b1c8-d24e6be1293d/manager/0.log"
Sep 10 14:49:08.143566 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:08.143538    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-tgw8z_fd18ebe1-c2ac-4a4a-8564-836411fd2a2a/kube-auth-proxy/0.log"
Sep 10 14:49:09.260124 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:09.260086    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-td7j4_fe39854b-f12d-4880-9f7c-fba9b24a8eb5/main/0.log"
Sep 10 14:49:09.363006 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:09.362981    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fn468d_c60385e0-2eb1-4ccf-ba76-0e4e8750126a/main/0.log"
Sep 10 14:49:09.569362 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:09.569264    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-lwvmz_e239519a-53dd-4eed-af49-3afa5463d150/main/0.log"
Sep 10 14:49:14.876684 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:14.876648    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:16.363429 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:16.363394    2576 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:18.418552 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:18.418519    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qwlw4_652327f0-1aef-4f03-b450-d89fcacee365/global-pull-secret-syncer/0.log"
Sep 10 14:49:18.463880 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:18.463838    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-57w2l_653fae07-2217-4639-89e2-465c3eea1bc7/konnectivity-agent/0.log"
Sep 10 14:49:18.552024 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:18.551992    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-41.ec2.internal_b604e50155af53c6f49e2fae328d06fe/haproxy/0.log"
Sep 10 14:49:22.688842 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:22.688746    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-p5n2j_e361ffe9-56a4-4a89-8f2e-fa37702ae7ae/manager/0.log"
Sep 10 14:49:22.716144 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:22.716119    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-mt5wl_c846bd41-86da-4f04-a5f9-7aedf9f95e0a/manager/0.log"
Sep 10 14:49:22.745711 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:22.745690    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-gh5th_ee23449c-2560-4f76-96fa-57b77cd7a0e0/kuadrant-console-plugin/0.log"
Sep 10 14:49:22.792940 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:22.792889    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-j2wb6_7effa53a-5c45-401d-82fb-e3be4b577cb2/manager/0.log"
Sep 10 14:49:22.839452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:22.839428    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-c57jv_f036dde9-ef00-417a-9b32-e0cb38ca8d04/limitador/0.log"
Sep 10 14:49:22.901840 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:22.901795    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-pmdgx_1f8014f5-448d-4fcd-b1c8-d24e6be1293d/manager/0.log"
Sep 10 14:49:24.967919 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:24.967887    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9g74h_fac3e327-1d2f-4765-8246-8f0c4c7a0917/node-exporter/0.log"
Sep 10 14:49:24.988931 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:24.988909    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9g74h_fac3e327-1d2f-4765-8246-8f0c4c7a0917/kube-rbac-proxy/0.log"
Sep 10 14:49:25.009520 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:25.009492    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9g74h_fac3e327-1d2f-4765-8246-8f0c4c7a0917/init-textfile/0.log"
Sep 10 14:49:28.033612 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:28.033580    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/1.log"
Sep 10 14:49:28.038676 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:28.038651    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-qhh2g_af20a173-9bc9-4e04-9074-8aa13acadbd5/console-operator/2.log"
Sep 10 14:49:29.295512 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.295480    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-bnmsv_5238c56a-d8ba-4307-95d5-e17c1eb00933/volume-data-source-validator/0.log"
Sep 10 14:49:29.599755 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.599670    2576 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"]
Sep 10 14:49:29.600055 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.600043    2576 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2390c2c5-774c-4ac6-8864-35ada9f31d11" containerName="maas-api"
Sep 10 14:49:29.600098 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.600057    2576 state_mem.go:107] "Deleted CPUSet assignment" podUID="2390c2c5-774c-4ac6-8864-35ada9f31d11" containerName="maas-api"
Sep 10 14:49:29.600149 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.600140    2576 memory_manager.go:368] "RemoveStaleState removing state" podUID="2390c2c5-774c-4ac6-8864-35ada9f31d11" containerName="maas-api"
Sep 10 14:49:29.603427 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.603405    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.605742 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.605715    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cvj6w\"/\"openshift-service-ca.crt\""
Sep 10 14:49:29.606839 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.606817    2576 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cvj6w\"/\"default-dockercfg-k2q7n\""
Sep 10 14:49:29.606939 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.606840    2576 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cvj6w\"/\"kube-root-ca.crt\""
Sep 10 14:49:29.610397 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.610371    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"]
Sep 10 14:49:29.672385 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.672355    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-podres\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.672540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.672401    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-sys\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.672540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.672425    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-lib-modules\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.672540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.672459    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-proc\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.672540 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.672477    2576 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xth62\" (UniqueName: \"kubernetes.io/projected/08acc5f7-9b1b-463f-b6ae-67ef390726f7-kube-api-access-xth62\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773747 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773719    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-podres\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773767    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-sys\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773793    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-lib-modules\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773859    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-sys\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773894    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-proc\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773899    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-podres\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773889    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-lib-modules\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.773933 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773924    2576 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xth62\" (UniqueName: \"kubernetes.io/projected/08acc5f7-9b1b-463f-b6ae-67ef390726f7-kube-api-access-xth62\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.774179 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.773986    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/08acc5f7-9b1b-463f-b6ae-67ef390726f7-proc\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.781906 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.781876    2576 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xth62\" (UniqueName: \"kubernetes.io/projected/08acc5f7-9b1b-463f-b6ae-67ef390726f7-kube-api-access-xth62\") pod \"perf-node-gather-daemonset-jvbfp\" (UID: \"08acc5f7-9b1b-463f-b6ae-67ef390726f7\") " pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:29.913813 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:29.913714    2576 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:30.183085 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:30.183058    2576 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"]
Sep 10 14:49:30.185662 ip-10-0-128-41 kubenswrapper[2576]: W0910 14:49:30.185630    2576 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod08acc5f7_9b1b_463f_b6ae_67ef390726f7.slice/crio-322836278d32d4e39d72886dd69889686949fe9485fa1f4c9e006b9979cb41bd WatchSource:0}: Error finding container 322836278d32d4e39d72886dd69889686949fe9485fa1f4c9e006b9979cb41bd: Status 404 returned error can't find the container with id 322836278d32d4e39d72886dd69889686949fe9485fa1f4c9e006b9979cb41bd
Sep 10 14:49:30.202017 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:30.201993    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp" event={"ID":"08acc5f7-9b1b-463f-b6ae-67ef390726f7","Type":"ContainerStarted","Data":"322836278d32d4e39d72886dd69889686949fe9485fa1f4c9e006b9979cb41bd"}
Sep 10 14:49:30.602194 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:30.602109    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bxrf8_da042357-c42d-4a96-970c-1ad9e2bbee80/dns/0.log"
Sep 10 14:49:30.626712 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:30.626692    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bxrf8_da042357-c42d-4a96-970c-1ad9e2bbee80/kube-rbac-proxy/0.log"
Sep 10 14:49:30.769446 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:30.769422    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4lmh_7b327197-ec2d-4799-9164-bbea8925c918/dns-node-resolver/0.log"
Sep 10 14:49:31.209063 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:31.209017    2576 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp" event={"ID":"08acc5f7-9b1b-463f-b6ae-67ef390726f7","Type":"ContainerStarted","Data":"398a6c10b747da87f115551960711ab6e0d742c45402bc2a21aa6b047d711f43"}
Sep 10 14:49:31.209305 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:31.209130    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:31.225704 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:31.225655    2576 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp" podStartSLOduration=2.225639808 podStartE2EDuration="2.225639808s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:49:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:49:31.223936866 +0000 UTC m=+2253.312394436" watchObservedRunningTime="2026-09-10 14:49:31.225639808 +0000 UTC m=+2253.314097378"
Sep 10 14:49:31.303294 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:31.303264    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5ccc597495-spmwc_7f7b1f52-033f-4de4-9d35-d094b670e283/registry/0.log"
Sep 10 14:49:31.332916 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:31.332895    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4spz6_c22a5c55-fbd5-4a43-bf63-0ec6b132b0be/node-ca/0.log"
Sep 10 14:49:32.603351 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:32.603317    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-tgw8z_fd18ebe1-c2ac-4a4a-8564-836411fd2a2a/kube-auth-proxy/0.log"
Sep 10 14:49:33.665245 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:33.665201    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-xs8p9_f456e19b-d821-4774-9c39-ce6bc666bc6d/serve-healthcheck-canary/0.log"
Sep 10 14:49:34.230021 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:34.229990    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gzp7p_3842435a-3a63-4c3d-b98b-a6f15b4e53fe/kube-rbac-proxy/0.log"
Sep 10 14:49:34.251564 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:34.251540    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gzp7p_3842435a-3a63-4c3d-b98b-a6f15b4e53fe/exporter/0.log"
Sep 10 14:49:34.272820 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:34.272795    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-gzp7p_3842435a-3a63-4c3d-b98b-a6f15b4e53fe/extractor/0.log"
Sep 10 14:49:36.539481 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:36.539447    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-qw6zk_74b7d555-ab98-434a-b5b7-95688e34645a/manager/0.log"
Sep 10 14:49:36.709396 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:36.709361    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mx5dx_243380c0-5722-40b2-bf45-727be8411511/manager/0.log"
Sep 10 14:49:36.801509 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:36.801433    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-dj2ng_90ece399-7522-4544-8c12-60067c925e44/server/0.log"
Sep 10 14:49:37.050628 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:37.050595    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-qgt9h_870bd46a-42b6-4a36-a639-f948e19bc0aa/manager/0.log"
Sep 10 14:49:37.071916 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:37.071849    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55f59f54dd-qgt9h_870bd46a-42b6-4a36-a639-f948e19bc0aa/copy-manifests/0.log"
Sep 10 14:49:37.224815 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:37.224786    2576 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cvj6w/perf-node-gather-daemonset-jvbfp"
Sep 10 14:49:38.704491 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:38.704468    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-jtsvr_0662801c-8dd3-4ffa-9ef2-4d3a556e8a83/openshift-lws-operator/0.log"
Sep 10 14:49:39.032727 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:39.032689    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-tgw8z_fd18ebe1-c2ac-4a4a-8564-836411fd2a2a/kube-auth-proxy/0.log"
Sep 10 14:49:40.987396 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:40.987372    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85dcd64b9c-tgw8z_fd18ebe1-c2ac-4a4a-8564-836411fd2a2a/kube-auth-proxy/0.log"
Sep 10 14:49:43.908219 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:43.908189    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8bwt8_bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64/migrator/0.log"
Sep 10 14:49:43.930709 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:43.930681    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-8bwt8_bc8f4b28-79a9-4f5d-bb9f-9c95b71fdb64/graceful-termination/0.log"
Sep 10 14:49:45.315333 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.315302    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/kube-multus-additional-cni-plugins/0.log"
Sep 10 14:49:45.337766 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.337742    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/egress-router-binary-copy/0.log"
Sep 10 14:49:45.359763 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.359739    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/cni-plugins/0.log"
Sep 10 14:49:45.379712 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.379690    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/bond-cni-plugin/0.log"
Sep 10 14:49:45.402847 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.402830    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/routeoverride-cni/0.log"
Sep 10 14:49:45.426878 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.426851    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/whereabouts-cni-bincopy/0.log"
Sep 10 14:49:45.448760 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.448739    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hwkkb_9b6fcde7-26f5-4234-a09a-f762fcdac8e4/whereabouts-cni/0.log"
Sep 10 14:49:45.869783 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.869751    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-qg85n_00231867-05ed-4f09-8495-092d8322939e/kube-multus/0.log"
Sep 10 14:49:45.990897 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:45.990866    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sr27l_745510db-ad2d-4b22-97b7-4224f10153d5/network-metrics-daemon/0.log"
Sep 10 14:49:46.009149 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:46.009122    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-sr27l_745510db-ad2d-4b22-97b7-4224f10153d5/kube-rbac-proxy/0.log"
Sep 10 14:49:47.358002 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.357973    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/ovn-controller/0.log"
Sep 10 14:49:47.379431 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.379399    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/ovn-acl-logging/0.log"
Sep 10 14:49:47.404559 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.404537    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/kube-rbac-proxy-node/0.log"
Sep 10 14:49:47.425452 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.425432    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/kube-rbac-proxy-ovn-metrics/0.log"
Sep 10 14:49:47.448596 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.448573    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/northd/0.log"
Sep 10 14:49:47.470394 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.470367    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/nbdb/0.log"
Sep 10 14:49:47.491781 ip-10-0-128-41 kubenswrapper[2576]: I0910 14:49:47.491756    2576 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-tnzpc_7f0a113a-476d-48e3-b4c4-5b4db4b11f54/sbdb/0.log"